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AOI 

area  of  interest 

CERCLA 

Comprehensive  Environmental  Response,  Compensation,  and  Eiability  Act 

CSM 

conceptual  site  model 

DBT 

Demolition  Bombing  Target 

DEM 

digital  elevation  model 

DoD 

Department  of  Defense 

EM 

electromagnetic 

EMI 

electromagnetic  induction 

ESTCP 

Environmental  Security  Technology  Certification  Program 

CIS 

Geographical  Information  System 

GPS 

Global  Positioning  System 

HE 

high  explosive 

IMU 

Inertial  Measurement  Unit 

lidar 

light  detection  and  ranging 

MEC 

munitions  and  explosives  of  concern 

MRS 

munitions  response  site 

NDIA 

New  Demolitions  Impact  Area 

PBR 

Precision  Bombing  Range 

QA 

quality  assurance 

QC 

quality  control 

SAR 

synthetic  aperture  radar 

SORT 

Simulated  Oil  Refinery  Target 

TIN 

Triangulated  Irregular  Network 

TRSI 

Terra  Remote  Sensing,  Inc. 

UXO 

unexploded  ordnance 

WAA 

wide  area  assessment 
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ABSTRACT 


URS  Corporation  and  Terra  Remote  Sensing,  Inc.  were  awarded  ESTCP  funding  to  demonstrate 
the  utility  of  high-density  lidar  and  orthophotography  as  one  component  of  a  multi-technology 
approach  to  UXO/munitions  and  explosives  of  concern  (MEC)  Wide  Area  Assessment  (WAA). 
Eidar  and  orthophotography  were  collected  at  two  former  military  sites:  the  5,000-acre  Kirtland 
Precision  Bombing  Range  site  near  Albuquerque,  New  Mexico,  and  the  5,640  Victorville 
Demonstration  Bombing  Target  (DBT)  Y  site  near  Victorville,  California.  Data  was  collected  at 
the  Kirtland  site  over  three  days  in  August  2005  and  at  the  Victorville  site  over  two  three-day 
periods  in  January  and  Eebruary  2006. 

Performance  objectives  of  the  demonstration  were  to: 

•  Identify  munitions  response  sites  (MRS)  such  as  bombing  targets,  open  burn/open 
detonation  areas,  and  burial  pits  that  are  the  result  of  sanctioned  military  activities 
(whether  documented  or  undocumented)  that  could  reasonably  be  expected  to  result 
in  the  release  of  MEC  to  the  environment 

•  Provide  information  about  the  site  and  the  MRS  to  support  future  investigation, 
prioritization,  and  cost  estimation 

•  Eor  areas  outside  of  the  MRS,  provide  information  to  support  regulatory  decisions 
regarding  the  portions  of  the  munitions  response  areas  outside  of  the  MRS,  including 
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decisions  as  to  requirements  for  further  investigation,  institutional  controls,  or  no 
further  action 

•  Describe  the  certainty  associated  with  the  initial  conceptual  site  model  (CSM),  and 
examine  the  incremental  contributions  of  each  technology  to  improvements  in  that 
certainty 

These  objectives  were  met.  The  MRS  and  munitions-related  features  were  successfully 
identified,  and  findings  were  used  to  verify  and  correct  the  initial  CSM.  Information  from  the 
demonstration  was  successfully  used  by  subsequent  demonstrators  and  ESTCP  in  subsequent 
phases  of  the  WAA  Pilot  Program.  All  data  accuracy  specifications  were  met.  The 
demonstration  provided  important  insights  regarding  the  appropriate  uses  and  confidence  levels 
for  both  lidar  and  orthophoto  technologies. 

At  the  Kirtland  site,  all  four  of  the  known  targets  from  the  CSM  were  identified  using  lidar  data. 
Additionally,  three  sub-target  areas  and  16  additional  areas  of  interest  were  identified,  including 
a  potential  bull’s-eye  target  ring  not  documented  in  the  CSM.  The  areas  of  interest  consisted  of 
isolated  groups  of  potential  craters. 

Bombing  targets  at  the  Kirtland  site  were  constructed  from  berms;  these  had  weathered  to 
between  5  cm  and  15  cm  (2  and  7  inches)  in  height,  and  most  were  not  visible  to  ground  crews 
on  the  site.  These  targets  were  clearly  visible  in  the  lidar  data  but  generally  not  in  the 
orthophotos.  However,  the  orthophotos  did  show  some  target  elements,  such  as  target  cross¬ 
hairs,  that  were  not  visible  in  the  lidar  data. 

Subsequent  validation  activities  using  magnetometry  and  intrusive  investigation  verified  the 
bombing  targets  and  the  three  ancillary  target  areas,  and  indicated  that  the  16  additional  areas  of 
interest  likely  were  not  munitions-related.  Validation  activities  also  showed  the  presence  of  two 
areas  of  concentrated  munitions-related  scrap  near  one  of  the  targets  that  were  not  revealed  by 
either  the  lidar  or  orthophoto  data. 

At  the  Victorville  site,  both  targets  identified  in  the  initial  CSM  were  visible  in  both  the  lidar  and 
orthophoto  data.  Large  craters  were  visible  in  and  near  DBT  Y,  the  Means  Dry  Lake  bed. 
Potential  craters  ranged  from  5  to  8  m  in  diameter  and  up  to  1  m  deep.  Potential  crater  locations 
were  used  to  modify  the  boundary  of  the  MRS  presented  in  the  CSM.  Target  PBR  15,  a  bull’s- 
eye  target  used  for  precision  bombing  practice,  consisted  of  a  series  of  rings  constructed  from 
asphalt.  The  target  area  was  clearly  visible  in  the  orthophoto  and  lidar  intensity  data,  and 
showed  no  evidence  of  craters. 

Increasing  the  lidar  data  density  increased  the  rate  of  detection  of  craters  in  the  1  m  to  3  m  size 
range  substantially,  but  had  only  a  weak  effect  on  detection  of  larger  craters,  additional  areas  of 
interest,  and  bombing  targets.  Lidar  density  effects  were  most  pronounced  between  the 
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2  2 

1.5pts./m  data  and  the  4.5  pts./m  data  sets,  and  diminished  thereafter.  Deteetion  of  faint  linear 
features  was  affeeted  by  flight  line  orientation.  Common  lidar  surface  artifacts  were 
encountered,  consisting  of  faint  lines  approximately  0.5  m  in  height. 

The  probability  of  detection  for  the  bombing  targets  at  both  sites  was  100%,  with  all  of  the 
known  targets  from  the  CSM  being  detected  at  both  sites.  Nevertheless,  MRS  at  the  Kirtland  site 
were  missed:  validation  activities  revealed  two  sites  with  concentrated  ordnance-related  scrap 
that  were  not  identified  using  either  lidar  or  orthophotos,  presumably  because  they  left  little  or  no 
surface  disturbance.  No  clear  false  alarms  were  encountered  for  bombing  targets.  There  was  no 
attempt  to  measure  false  alarm  rates  for  individual  potential  craters. 

Overall  costs  for  acquisition  and  processing  of  lidar  and  orthophotos  were  approximately 
$48/acre  for  Kirtland  and  $27/acre  for  Victorville.  Per-acre  costs  at  Kirtland  were  higher  due  to 
acquisition  of  four  lidar  flights  and  two  orthophoto  sets,  compared  to  two  lidar  flights  and  one 
orthophoto  set  at  Victorville.  Costs  are  projected  to  be  approximately  $20/acre  for  an 
18,000-acre  site,  based  on  preliminary  results  from  the  Former  Camp  Beale  site  acquired  in  late 
2006.  Costs  are  projected  at  between  $12  and  $14/acre  for  sites  in  the  50,000  -  1 10,000  acre 
range.  Costs  for  competing  technologies  are  discussed  in  the  Final  Report  for  the  WAA  Pilot 
Program. 

The  results  of  the  demonstration  support  the  use  of  lidar  and  orthophotos  as  an  integral  part  of 
the  WAA  Process.  These  technologies  proved  to  be  a  cost-effective  and  reliable  means  to 
characterize  the  sites,  validate  and  correct  the  initial  CSMs,  and  provide  data  to  focus  the 
application  of  subsequent  methods  of  investigation.  Lidar  and  orthophotos  will  not  detect  sites 
where  there  are  no  surface  indications.  Consequently,  use  of  lidar  and  orthophotos  should  be 
followed  by  technologies  that  detect  munitions  components  directly. 
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1,0  INTRODUCTION 


1.1  BACKGROUND 

Many  millions  of  acres  of  Department  of  Defense  (DoD)  lands  are  potentially  contaminated  with 
military  munitions  or  their  components.  On  the  majority  of  these  sites,  munitions  are 
concentrated  in  specific  ranges  and  training  areas,  while  the  remainder  of  the  site  is  ordnance- 
free.  Contaminated  sites  traditionally  have  been  very  expensive  to  investigate  and  remediate 
beeause  of  the  nature  of  the  contamination  and  the  relatively  few  innovative  approaehes 
available  to  date. 

The  eurrent  demonstration  was  eondueted  as  part  of  the  Environmental  Security  Technology 
Certification  Program  (ESTCP)  Wide  Area  Assessment  (WAA)  Pilot  Program,  which  explored 
the  use  of  an  integrated  suite  of  airborne  and  ground-based  teehnologies  as  a  means  to  identify 
and  validate  pilot  WAA  teehnologies  and  approaehes.  Eight  deteetion  and  ranging  (lidar)  and 
orthophotography,  the  subjeets  of  this  demonstration,  were  used  in  conjunetion  with  synthetie 
aperture  radar  (SAR),  hyperspeetral  sensing,  helicopter-based  magnetometry,  and  towed-array 
magnetometry  and  eleetromagnetie  induetion  (EMI),  along  with  statistieal  modeling,  in  an 
integrated  Geographieal  Information  System  (GlS)-based  analytical  environment. 

This  report  diseusses  the  results  of  lidar  and  orthophoto  data  collection  and  analysis  at  two 
demonstration  sites.  The  results  from  the  WAA  Pilot  Program  as  a  whole,  and  the  relationship  of 
these  two  teehnologies  to  the  entire  suite  of  teehnologies  tested,  are  diseussed  in  the  final  report 
for  the  WAA  Pilot  Program  as  a  whole. 


1.2  OBJECTIVES  OF  THE  DEMONSTRATION 

The  objective  of  the  demonstration  was  to  demonstrate  and  validate  the  ability  of  lidar  and 
orthophotos  to  contribute  to  the  WAA  proeess  by: 

•  Identifying  munitions  response  sites  (MRS)  such  as  bombing  targets,  open  burn/open 
detonation  areas,  and  burial  pits  that  are  the  result  of  sanctioned  military  activities 
(whether  doeumented  or  undoeumented)  that  could  reasonably  be  expected  to  result 
in  the  release  of  munitions  and  explosives  of  eoneem  (MEC)  to  the  environment 

•  Providing  information  about  the  site  and  the  MRS  to  support  future  investigation, 
prioritization,  and  cost  estimation 

•  Eor  areas  outside  of  the  MRS,  providing  information  to  support  regulatory  deeisions, 
ineluding  decisions  as  to  requirements  for  further  investigation,  institutional  eontrols, 
or  no  further  aetion 
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•  Describing  the  certainty  associated  with  the  initial  conceptual  site  model  (CSM),  and 
examining  the  incremental  contributions  of  each  technology  to  improvements  in  that 
certainty 

An  additional  objective  was  to  develop  information  about  the  factors  that  would  affect  the  cost 
and  performance  of  both  technologies,  including  the  relationship  between  levels  of  effort  and 
confidence  in  conclusions.  Data  density  was  the  primary  performance  factor  tested. 


1.3  REGULATORY  DRIVERS 

MEC  remediation  is  generally  conducted  under  authority  of  the  Comprehensive  Environmental 
Response,  Compensation,  and  Eiability  Act  (CERCLA).  With  many  millions  of  acres  of  land 
potentially  contaminated  with  MEC,  estimates  of  the  cost  of  elimination  of  environmental 
liability  under  this  statute  at  known  and  former  DoD  sites  range  as  high  as  several  hundred 
billion  dollars.  These  potentially  high  costs  have  led  to  interest  in  the  development  of  innovative 
investigative  or  screening  methods,  in  order  to  reduce  the  costs  of  conducting  WAA  and 
associated  remediation  activities. 


1.4  STAKEHOLDER/END-USER  ISSUES 

The  demonstration  showed  that  both  lidar  and  orthophotos  can  contribute  to  the  WAA  process 
through  cost-effective  delineation  of  MRS-  and  MEC-related  features.  The  demonstration  results 
indicate  that  lidar  and  orthophotos  are  most  appropriately  used  during  the  early  phases  of  the 
WAA  process,  in  order  to  focus  and  prioritize  the  subsequent  application  of  more  expensive  low- 
altitude  and  ground-based  technologies. 
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2.0  TECHNOLOGY  DESCRIPTION 

2.1  TECHNOLOGY  DEVELOPMENT  AND  APPLICATION 

2.1.1  Technology  Background 

Lidar  is  a  well-established  airborne  teehnology  for  modeling  ground  surfaees.  Topographie  lidar 
was  first  developed  in  the  late  1960s  and  early  1970s,  and  has  been  used  for  terrain  profiling 
sinee  the  mid-1980s.  Lidar  has  been  in  wide  commercial  use  since  around  1993,  and  the 
accuracies  and  limitations  of  lidar  for  surface  modeling  are  well  documented. 

Lidar  uses  the  time  of  return  for  a  laser  pulse  to  be  reflected  back  to  the  sensor  to  measure  the 
elevation  of  the  point  of  reflection.  Use  of  Global  Positioning  System  (GPS)  and  Inertial 
Measurement  Unit  (IMU)  technology  to  locate  the  sensor  precisely  in  the  air  allows  for  the 
accurate  calculation  of  the  point  of  reflection  of  the  laser  signal  from  the  ground,  buildings  or 
vegetation.  Multiple  returns  from  a  single  laser  pulse  can  be  detected,  increasing  the  chance  of 
sampling  the  ground  surface  through  gaps  in  vegetation.  Once  elevation  data  is  collected  in  the 
form  of  lidar  points,  surface  models  are  created  and  analyzed.  The  surface  modeling  process  is 
typically  conducted  using  standard  GIS  software  and  methods,  and  much  of  the  process  can  be 
successfully  automated.  Lidar  vendors  typically  guarantee  a  vertical  accuracy  of  0.15  m  and  a 
horizontal  accuracy  of  0.3-0.75  m. 


Figure  2-1 

Helicopter-Mounted  Lidar  and  Orthophoto  Sensor  Equipment 
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Figure  2-2 

Lidar  System  Operations 


The  development  of  higher-speed  (50-100  kHz)  laser  scanners  beginning  around  2002  has 
significantly  improved  the  ability  of  lidar  to  locate  small  features.  Currently,  high-speed  lidar 
systems  are  being  used  to  characterize  objects  in  the  sub-meter  range,  such  as  power  line 
insulators.  The  accuracy  and  data  density  of  current  lidar  systems  suggest  that  the  technology 
could  be  used  to  detect  ground  features  indicative  of  ordnance  use,  including  targets  and  craters, 
and  that  the  presence  of  these  features  could  in  turn  be  used  to  develop  more  accurate  locations 
ofMEC. 

Digital  orthophotography  has  been  commercially  available  since  the  early  1980s,  with  steady 
improvement  in  the  resolution  (i.e.,  pixel  size)  and  precision  (i.e.,  pixel  placement)  of  the  images 
as  the  technology  of  digital  cameras,  GPS,  and  IMU  systems  has  advanced.  Since  the  mid 
1990s,  image  size  has  advanced  from  1,500  pixels  across  an  image  to  4,500  pixels.  This  has 
allowed  for  increased  flying  heights  and  a  reduced  number  of  images  for  a  given  area,  with 
consequent  cost  savings.  Commensurate  with  this  improvement  has  been  a  twofold  increase  in 
the  accuracy  of  the  IMU,  allowing  for  accurate  positioning  of  image  pixels  at  a  higher  flying 
height. 
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Airborne  digital  cameras  have  been  successfully  integrated  with  lidar  sensors.  Cameras  with  an 
image  density  of  roughly  4,000  x  4,000  pixels  are  generally  favored,  because  the  width  of  the 
images  collected  is  very  similar  to  that  of  the  typical  lidar  point  swath.  Once  collected, 
individual  digital  images  are  mosaiced  and  color  balanced,  and  the  resulting  composite  image  is 
orthorectified  using  the  lidar  data.  Orthorectification  allows  for  the  accurate  location  of  each 
photo  pixel,  eliminating  distortion  caused  by  camera  angle  and  topography.  Vendors  generally 
guarantee  a  horizontal  accuracy  of  3  pixel  widths  compared  to  ground  control  for 
orthophotography. 

Digital  images  are  collected  concurrently  with  lidar  and,  because  the  two  sensors  use  the  same 
GPS  and  IMU,  the  two  data  sets  can  be  integrated  very  accurately.  Vendors  generally  guarantee 
spatial  integration  of  orthophotos  and  lidar  within  2  pixel  widths.  Final  orthophoto  pixel  size 
depends  on  the  flight  altitude  and  the  camera  specifications;  helicopter-based  cameras  flying  at 
altitudes  of  400-450  meters  are  capable  of  pixel  sizes  of  approximately  10  cm.  Smaller  pixel 
sizes  than  this  are  generally  impractical  due  to  the  low  flight  elevations  and  slow  flight  speeds 
required  to  collect  properly  overlapping  images,  and  the  very  large  numbers  of  images  that 
would  need  to  be  mosaiced. 

The  ability  to  produce  spatially  accurate  orthophotos  with  relatively  small  pixel  sizes  suggests 
that  this  technology  could  be  used  to  identify  ordnance-related  features,  and  to  cross-validate 
technologies  such  as  lidar. 


2.2  PREVIOUS  TESTING  OF  THE  TECHNOLOGY 

URS  and  Terra  Remote  Sensing,  Inc.  (TRSI)  conducted  a  successful  demonstration  of  lidar  and 
high-resolution  digital  imagery  at  an  operational  US  Navy  range  near  Boardman,  Oregon  during 
November  2004.  Both  the  lidar  data  and  the  orthophotos  were  successful  in  detecting  patterns  of 
surface  disturbance  indicative  of  unexploded  ordnance  (UXO)/MEC  activities.  Moreover,  the 
two  data  sets  were  found  to  complement  each  other  well,  each  revealing  features  that  the  other 
did  not.  Orthophotos  collected  during  the  November  2004  test  flight  showed  color  variations 
that  could  not  be  detected  by  lidar  (for  example,  painted  truck  tires  used  to  outline  bombing 
targets).  Very  subtle  ground  features,  such  as  vehicle  tracks,  which  were  beyond  the  resolution 
of  the  lidar  data  at  that  resolution,  were  visible  in  the  photos. 

Even  though  the  lidar  data  was  collected  at  relatively  low  density,  lidar  revealed  depressions 
such  as  craters  better  than  the  orthophotos,  where  it  was  difficult  to  distinguish  depressions  from 
shadows  or  mounds.  Eidar  was  very  successful  at  locating  target  features  such  as  bull’s-eye 
rings.  Disposal  craters  could  be  distinguished  from  bombing  practice  craters  by  their  patterns  on 
the  ground.  Surface  models  created  from  lidar  data  could  be  analyzed  in  ways  that  orthophoto 
data  could  not  be,  such  as  measuring  the  depth  of  craters. 
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2,3  FACTORS  AFFECTING  COST  AND  PERFORMANCE 

Factors  affecting  cost  and  performance  of  lidar  and  orthophotos  are  detailed  in  Table  2-1. 


Table  2-1 

Cost  and  Performance  Factors 


Item 

Cost  and  Performance  Factors 

Lidar  data  density 

Higher  lidar  data  density  is  generally  more  expensive,  due  to  the  need  to  aequire  and 
process  additional  flight  lines. 

Orthophoto  data 
density 

Orthophotos  with  smaller  pixels  are  more  expensive,  because  of  the  additional  flight  lines 
required  and  the  larger  number  of  individual  digital  images  that  must  be  processed  and 
mosaiced. 

Aeeuraey  and 

preeision 

requirements 

Accuracy  and  precision  are  largely  determined  by  the  equipment  used  and  the  care  of  the 
operators.  Projects  with  extremely  high  accuracy  requirements,  such  as  creation  of  contour 
lines  under  1-foot  intervals,  can  only  be  accomplished  by  vendors  with  extremely  new 
equipment,  at  higher  cost. 

Site  loeation  and 
logisties 

Sites  with  longer  flying  times  to  an  airport  will  be  more  expensive,  as  they  will  require 
either  longer  flight  times  or  placing  fuel  on  the  test  site. 

Aeeuraey 

verifieation 

requirements 

Verification  of  accuracy  and  precision  is  accomplished  through  placement  of  survey 
control  points  and  comparison  of  lidar  and  orthophoto  data  to  those  points.  Projects  with 
higher  verification  requirements  will  be  more  costly,  although  this  factor  is  small  compared 
to  other  factors. 

Site  size 

Larger  sites  can  achieve  substantial  cost  savings  through  amortization  of  fixed  costs,  such 
as  mobilization  and  project  planning,  as  well  as  through  increased  efficiency  in  data 
acquisition  and  processing. 

Vegetation 

eonditions 

More  densely  vegetated  sites  will  have  higher  costs  due  to  the  requirement  for  additional 
lidar  passes  to  achieve  sufficient  point  density  at  the  ground  surface.  Vegetation  will  affect 
the  ability  of  both  lidar  and  orthophotos  to  view  or  model  the  ground  surface. 

Permitting  and  site 
access 

Some  DoD  sites  contain  high-security  areas,  which  can  present  higher  costs  for  pre-flight 
planning  and  data  collection.  These  costs  result  from  restrictions  on  site  access,  time  to 
acquire  needed  clearances,  and  longer  flight  times  to  avoid  restricted  areas. 

Sites  with  environmental  constraints  do  not  normally  impose  higher  costs  for  lidar  and 
orthophotography,  due  to  the  airborne  nature  of  the  technologies.  However,  the  presence 
of  sensitive  species  may  affect  pre-flight  planning  and  scheduling  (and  thus  costs)  for 
projects  that  require  landing  to  re-fuel  on  the  site. 
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2,4  ADVANTAGES  AND  LIMITATIONS  OF  THE  TECHNOLOGY 

Advantages  and  limitations  offered  by  use  of  lidar  and  orthophotos  eompared  to  the  traditional 
approaches  to  MRS  investigation  as  shown  on  Table  2-2. 


Table  2-2 

Technology  Advantages  and  Limitations 


Item 

Advantages 

Rate  of  eoverage 

In  an  operational  setting,  data  colleetion  rates  of  5,000  acres  per  day  can  be  expected  for 
lidar  and  orthophotos.  This  compares  favorably  to  maximum  collection  rates  of  around 

500  acres  per  day  for  helicopter-based  magnetometry,  and  20  acres  per  day  for  towed-array 
magnetometry. 

Ability  to  delineate 
MRS  and  MEC- 
related  features 

Lidar  and  orthophotography  successfully  revealed  MRS  and  MEC-related  surface  features 
at  both  demonstration  sites,  even  many  years  after  their  last  use.  In  the  case  of  the  Kirtland 
site,  these  features  were  often  not  visible  to  observers  on  the  ground. 

Enhaneed  planning 
and  risk  assessment 

Because  they  can  cover  entire  sites  relatively  quickly  and  at  lower  cost,  these  technologies 
can  be  used  to  locate  and  prioritize  appropriate  areas  for  use  of  more  costly  ground-based 
technologies. 

Other  benefits 

Both  technologies  provide  highly  detailed  topographic  data  that  can  be  integrated  into  a 
facility’s  CAD  or  GIS  system  and  used  in  subsequent  phases  of  site  investigation,  site 
remediation,  and  range  management 

Item 

Limitations 

MEC  deteetion 

Neither  lidar  nor  orthophotography  can  directly  detect  small  dimension  shell  casings  or 
other  MEC  components.  Consequently,  further  investigation  with  magnetometers  or 
electromagnetic  (EM)  sensors  is  generally  required. 

Elevation  data 

Orthophotos  do  not  contain  elevation  information.  In  practice,  it  is  sometimes  difficult  to 
distinguish  small  surface  depressions  from  small  mounds  or  shadows  using  orthophotos 
alone. 

Vegetation  effeets 

Since  both  lidar  and  orthophotos  are  light-based  technologies,  neither  will  penetrate 
vegetation.  Orthophotos  do  not  “look  through”  vegetation,  and  lidar  point  densities  will  be 
lower  in  vegetated  areas.  However,  lidar  is  frequently  successful  in  penetrating  small 
openings  between  and  within  vegetation,  and  this  success  has  increased  with  the  speed  of 
lidar  sensors  and  the  development  of  the  ability  to  measure  multiple  returns. 
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3.0  DEMONSTRATION  DESIGN 

3.1  PERFORMANCE  OBJECTIVES 

The  primary  performance  objectives  for  these  technologies  were  to; 

•  Clarify  whether  and  to  what  extent  lidar  and  orthophotos  can  delineate  MRS 
boundaries  and  MEC-related  features,  and  contribute  to  focusing  and  prioritizing 
subsequent  low-altitude  and  ground-based  work 

•  Reveal  relationships  between  the  density  of  lidar  and  orthophoto  data,  their  levels  of 
cost,  and  their  ability  to  accurately  locate  MRS  boundaries  and  MEC-related  ground 
features 

•  Clarify  whether  and  to  what  extent  lidar  and  orthophotos  can  verify,  reveal  errors  in, 
or  improve  the  accuracy  of  the  initial  CSM 

•  Contribute  data  and  analysis  to  the  overall  combination  of  technologies  used  in  the 
WAA  Pilot  Program,  in  a  manner  that  is  timely  to  the  application  of  the  other 
technologies  demonstrated,  in  formats  useable  by  other  demonstrators,  and  with 
sufficient  positional  accuracy  compared  to  project  control  points  to  allow  meaningful 
coordination  and  comparison 

Specific  performance  criteria  and  performance  metrics  related  to  each  of  these  objectives  are 

established  in  the  Technology  Demonstration  Plan  for  each  site. 


3.2  SELECTION  OF  TEST  SITES 

Both  demonstration  sites  were  chosen  by  the  ESTCP  Program  Office.  Details  of  the  site 
selection  process  can  be  found  in  the  final  report  for  the  WAA  Pilot  Program. 


3.3  TEST  SITE  HISTORY,  CHARACTERISTICS  AND  PRESENT  OPERATIONS 

The  first  demonstration  site  was  located  at  the  Kirtland  Air  Force  Base  Precision  Bombing 
Range  (PBR)  located  approximately  10  miles  west  of  Albuquerque,  New  Mexico  (Figure  3-1). 
The  site  is  part  of  a  much  larger  set  of  bombing  ranges  used  for  training  purposes  during  World 
War  II.  The  study  site  consisted  of  approximately  5,120  acres  within  the  PBR  located  in  two 
parcels  to  the  north  and  south  of  the  Double  Eagle  Airport,  the  primary  small  aircraft  airport  for 
the  Albuquerque  area.  The  study  area  itself  is  currently  undeveloped,  although  portions  are 
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planned  for  commercial  or  industrial  development,  and  airport  expansion  into  the  study  are  is 
possible. 


Figure  3-1 

Kirtland  Precision  Bombing  Range  Site 


Legend 
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Munitions  known  or  suspected  to  have  been  used  at  the  site  include  100-point  practice  bombs 
and  250-pound  high  explosive  (HE)  bombs.  A  certificate  of  clearance  has  been  issued  for  one 
portion  of  the  site,  Target  N3.  Four  target  areas  were  identified  in  the  initial  CSM  as  within  the 
study  site: 


•  Target  N-2,  a  1,000-foot-diameter  bull’s-eye  target  used  for  100-pound  practice 
bombs 

•  Target  N-3,  a  1,000  -foot-diameter  bull’s-eye  target  used  for  100-pound  practice 
bombs  and  for  scrap  storage 

•  The  New  Demolitions  Impact  Area  (NDIA),  a  1,000  -foot-diameter  HE  bull’s-eye 
target 
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•  The  Simulated  Oil  Refinery  Target  (SORT),  a  target  consisting  of  350-foot  x  350-foot 
rectangular  cells 

Locations  were  given  for  the  first  three  targets  in  the  CSM.  The  approximate  location  of  the 
SORT  was  given  as  “somewhere  northwest  of  the  Double  Eagle  airport.” 

The  second  demonstration  site  was  located  at  the  Former  Victorville  Army  Air  Force  Demolition 
Bombing  Target  (DBT)  Y  and  PBR  15  (Figure  3-2).  The  site  is  located  in  San  Bernardino 
County,  California,  approximately  42  miles  southeast  of  the  town  of  Victorville,  California.  This 
site  lies  within  a  much  larger  complex  of  approximately  23  targets  used  between  1942  and  1945. 
The  site  is  managed  by  the  US  Bureau  of  Fand  Management,  and  is  used  primarily  as  a 
recreation  area  for  off-road  vehicles,  camping,  and  target  shooting.  The  demonstration  site 
encompasses  approximately  5,640  acres. 


Figure  3-2 

Victorville  DBT  Y  and  PBR  Target  15  Site 
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Two  target  areas  were  identified  in  the  initial  CSM  for  the  Victorville  site: 

•  Target  DBT  Y,  located  in  Means  Dry  Lake  bed  in  the  center  of  the  demonstration 
site,  identified  as  a  demolition  bomb  target  area  where  bombs  between  100  and  2,000 
pounds  were  used. 

•  Target  PBR  15,  a  suspected  bull’s-eye  target  located  in  the  southeast  portion  of  the 
demonstration  site  used  for  precision  bombing  practice.  According  to  the  initial 
CSM,  PBR  15  was  not  visited  during  the  archival  search  report  site  visit  and  little  is 
known  about  the  area. 


3.4  PRE-DEMONSTRATION  TESTING  AND  ANALYSIS 
3,4,1  Kirtland  Demonstration  Site 

Prior  to  mobilization,  ESTCP  established  five  survey  control  points  on  the  Kirtland  site 
(Figure  3-3).  TRSI  established  four  additional  control  points  and  independently  occupied  two  of 
the  ESTCP  control  points.  Six  vertical  control  structures  were  placed  adjacent  to  the  six  control 
points  established  by  TRSI,  but  two  were  vandalized  prior  to  data  collection.  A  test  crater  area 
was  established  in  the  south  portion  of  the  project  site  containing  two  test  craters  at  1.5  m 
diameter,  two  at  1.0  m  diameter,  and  six  at  0.32  m  diameter  (Figure  3-4). 


Figure  3-3 

Kirtland  Control  Points 


Control  point  locations 


Example  Kirtland  control  point 
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Figure  3-4 

Kirtland  Test  Craters 


0.32  m  test  eraters 


1.5  m  test  erater 


On  August  9,  2005,  calibration  flights  were  conducted  to  establish  appropriate  pitch,  yaw,  and 
roll  correction  factors  and  appropriate  lidar  sensor  speeds.  A  laser  pulse  rate  of  50  kHz  was 
selected,  based  on  performance  at  a  variety  of  speeds  tested.  Lidar  and  orthophoto  missions 
began  the  same  day. 

3,4,2  Victorville  Demonstration  Site 


At  the  Victorville  site,  TRSI  established  eight  survey  control  points  on  the  site  (Figure  3-5). 

These  sites  were  later  independently  oecupied  by  ESTCP.  Four  vertical  control  structures  and  10 
test  eraters  were  also  established,  at  the  same  sizes  as  for  the  Kirtland  site  (Figure  3-6).  Unlike  at 
Kirtland,  the  Victorville  test  craters  were  not  grouped  together  in  one  loeation.  This  revised 
configuration  eliminated  the  observed  tendency  to  infer  the  location  of  smaller  test  craters  from 
the  known  location  of  the  adjacent  larger  craters. 
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Figure  3-5 

Victorville  Control  Points 


Example  Vietorville  eontrol  point 


On  February  4,  2006,  calibration  flights  were  conducted.  A  laser  pulse  rate  of  40  kHz  was 
selected  as  most  appropriate  to  meet  overall  point  density  targets.  Orthophoto  flights  began  on 
January  24,  2006  and  lidar  acquisition  flights  began  on  February  3,  2006. 


Figure  3-6 

Victorville  Test  Craters 


H>  9 _  » _ » 

MilM« 


Sample  test  eraters 


Sample  test  eraters  and  vertieal  eontrol  strueture 
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3.5  TESTING  AND  EVALUATION  PLAN  AND  PROCEDURES 
3,5,1  Demonstration  Set-Up  and  Start-Up 

At  both  sites,  TRSI  conducted  flight  line  planning  for  the  lidar/orthophoto  and  lidar  only  flights 
in  the  weeks  prior  to  mobilization.  Flight  lines  were  planned  to  ensure  complete  site  coverage, 
minimize  the  number  of  turns,  and  achieve  planned  overlap.  At  the  Kirtland  site,  flight  lines 
were  planned  to  minimize  interactions  between  data  collection  and  air  traffic  at  the  Double  Eagle 
Airport.  Digital  imagery  was  planned  for  acquisition  at  periods  of  low  sun  angle  at  both  sites. 
Previous  testing  had  shown  that  the  shadows  created  by  low  sun  angle  were  useful  in  detecting 
shallow  features. 

At  each  site,  the  lidar  and  orthophoto  sensor  system  was  installed  into  a  Bell  206B  helicopter 
owned  by  a  local  helicopter  vendor  (Figure  3-7).  Renting  helicopters  (and  pilots)  using  local 
vendors  is  a  standard  industry  practice  which  allows  the  lidar  vendor  to  ship  only  the  sensor 
package  rather  than  the  aircraft.  The  use  of  local  helicopter  vendors  also  allows  for  the  use  of 
local  pilots  who  have  better  knowledge  of  local  weather  patterns  and  flight  clearance 
requirements. 


Figure  3-7 

Lidar  Equipment  Setup 


Equipment  installation  Sensor  pod  mounted  below  helicopter,  control 

console  visible  through  window 


3,5,2  Period  of  Operation  and  Area  Characterized 

Data  collection  flights  were  begun  once  mobilization,  sensor  installation,  and  calibration  flights 
were  complete.  The  period  for  data  collection  included  an  additional  day  to  allow  for  re¬ 
acquisition  of  any  missed  or  erroneous  areas  discovered  during  daily  quality  assurance/quality 
control  (QA/QC)  review.  Data  collection  periods  were  as  shown  on  Table  3-1. 
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Table  3-1 

Data  Collection  Periods 


Period 

Kirtland 

Victorville 

Preflight  planning 

July  2005 

January  2006 

Mobilization 

August  8,  2005 

February  7,  2006 

Calibration  flights 

August  8,  2005 

February  8,  2006 

Data  collection 

August  9-11,  2005 

January  24-25,  2006 
February  3-4,  2006 

Demobilization 

August  1,  2005 

February  5,  2006 

At  Kirtland,  the  area  eharaeterized  was  1,914  heetares  or  5,000  aeres.  At  Vietorville  the  area 
was  2,282  heetares  or  5,640  aeres. 

3.5.3  Demobilization 

Demobilization  of  the  lidar  and  orthophoto  equipment  eonsisted  of  demounting  the  sensor 
system  from  the  helieopter,  paeking  and  shipping.  Demobilization  required  approximately  a  half 
day  for  eaeh  site. 


3.6  SELECTION  OF  ANALYTICAL/TESTING  METHODS 

Analysis  of  lidar  data  is  performed  in  two  steps:  eonversion  of  sensor  output  to  spatially  eorreet 
lidar  points,  and  then  eonversion  of  these  points  to  useable  GIS  produets  sueh  as  surfaee  models. 
Lidar  data  was  proeessed  into  point  fdes  using  a  suite  of  software,  ineluding  TerraSolid  and 
eustom  algorithms  written  in  this  software  by  TRSl.  TerraSolid  is  an  industry  standard  software 
paekage  for  proeessing  lidar  data. 

Creation  of  GIS  produets  and  their  analysis  were  aeeomplished  using  ESRI’s  AreGlS  software 
suite.  This  paekage  was  ehosen  for  two  reasons.  First,  AreGlS  was  the  only  standard  CAD  or 
GIS  produet  reviewed  that  would  suoeessfully  handle  the  large  number  of  lidar  points  eolleeted. 
Seeond,  AreGlS  is  the  GIS  paekage  most  widely  used  by  US  government  ageneies  and  private 
eontractors.  As  sueh,  it  is  appropriate  to  develop  analysis  methods  and  resulting  products  that 
can  be  duplicated  by  typical  federal  facilities  managers. 

Orthophoto  data  was  analyzed  in  two  steps:  creation  of  a  single  orthorectified  image  from  the 
large  number  of  individual  digital  images  collected,  followed  by  visual  examination  of  the  image 
to  locate  potential  MRS  and  munitions-related  features.  Processing  of  the  digital  imagery  to 
create  the  orthophoto  mosaic  was  accomplished  using  software  from  TerraSolid  and  PCI.  This 
software  is  the  industry  standard. 
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3.7  RESULTS 
3,7,1  Data  Collection 


At  the  Kirtland  site,  data  eolleetion  flights  took  plaee  on  August  9,  10,  and  11,  2005  with  a  total 
of  177  flight  lines  eolleeted,  totaling  555  km.  Lidar  data  was  eollected  at  three  altitudes;  900 
meters,  450  meters,  and  300  meters  (Figure  3-8).  Two  sets  of  flight  lines  were  flown  at  300 
meters.  For  the  north  portion  of  the  projeet  area  the  300  meter  flight  lines  were  flown 
perpendieular  to  eaeh  other,  for  the  south  portion  they  were  flown  parallel.  Orthophoto  data  was 
eolleeted  eoneurrently  with  lidar  during  the  900  meter  and  450  meter  flights. 


Figure  3-8 

Kirtland  Sample  Flight  Lines 


Kirtland  flight  lines,  900  m  lidar  and  20  cm 
orthophotos 


Data  was  inspected  eaeh  evening  for  GPS  positional  accuracy,  completeness  of  lidar  data 
coverage  and  imagery  data  coverage.  During  the  final  afternoon  of  data  collection,  flight  lines 
that  did  not  meet  specifications  were  re-flown. 


At  the  Victorville  site,  orthophoto  data  collection  flights  took  place  on  January  24  and  25,  2006 
and  lidar  data  collection  flights  took  place  on  February  3  and  4,  2006.  A  total  of  45  flight  lines 
were  collected,  totaling  601  km.  Lidar  data  was  collected  at  two  altitudes:  450  meters  and  300 
meters  (Figure  3-9).  The  two  flights  were  flown  perpendicular  to  each  other.  As  at  the  Kirtland 
site,  data  was  inspected  each  evening  for  accuracy  and  completeness,  and  data  gaps  were  filled 
by  additional  flights  prior  to  demobilization. 


FINAL  REPORT 

High  Density  Lidar  and  Orthophotography 
in  UXO  Wide  Area  Assessment 
Contraet  No.  N44255-02-D-2008 


Seetion  4.0 
Revision  No.:  1 
Date:  August  2007 
Page  3-10 


Figure  3-9 

Victorville  Sample  Flight  Lines  and  Example  Image 


Vietorville  flight  lines,  300  m  lidar 
flight 


Vietorville  flight  lines,  450  -  m 
lidar  and  10  em  orthophotos 


1 

11 

B 

Example  image  from  software  used 
to  eheek  for  eomplete  eoverage  by 
digital  imagery,  450  m  flight 


3.7.2  Safety  Issues 

At  the  Kirtland  site,  an  important  safety  issue  was  the  high  level  of  air  traffic  at  the  Double  Eagle 
Airport.  This  airport  experiences  more  than  100,000  take-offs  per  year,  and  is  located 
immediately  adjacent  to  the  study  site  between  the  north  and  south  blocks.  In  response  to  the 
high  level  of  air  traffic,  a  second  pilot  was  hired  to  be  a  spotter  during  data  collection  flights. 

This  extra  weight  led  to  more  frequent  refueling  and  longer  data  collection  work  days.  The  air 
traffic  situation  also  prevented  the  acquisition  of  perpendicular  lidar  flight  lines  for  the  south 
block,  which  would  have  required  turning  directly  over  the  runway. 

3.7.3  Data  Processing  Steps 

TSRI  processed  the  sensor  output  to  create  lidar  points.  Following  return  of  the  data  to  the 
office,  calibration  factors  determined  in  the  field  were  checked,  fine-tuned,  and  applied  to  laser 
range,  GPS,  and  IMU  data  to  produce  x,y,z  values  for  each  point.  Lidar  points  were  then 
transformed  into  the  delivery  datum  and  projection  for  each  site,  and  coded  to  indicate  returns 
from  ground  vs.  non-ground  surfaces.  Lidar  points  were  exported  as  text  files  for  delivery  to 
URS.  Lidar  data  from  each  lidar  flight  was  processed  and  delivered  separately  in  order  to  allow 
for  separate  analysis  of  data  from  each  flight  altitude. 

TSRI  also  processed  digital  images.  The  procedure  included  mosaicing  the  individual  digital 
images  collected  during  flight,  transforming  the  consolidated  image  to  the  delivery  datum  and 
projection,  orthorectification  using  the  lidar  data,  color  balancing,  and  trimming  to  the  delivery 
tiles.  For  the  Kirtland  site,  two  sets  of  orthomosaics  were  created,  one  at  10  cm  pixel  size  and 
one  at  20  cm  pixel  size.  For  the  Victorville  site,  a  single  set  of  orthophotos  was  created  at  10  cm 
pixel  size. 


FINAL  REPORT 

High  Density  Lidar  and  Orthophotography 
in  UXO  Wide  Area  Assessment 
Contraet  No.  N44255-02-D-2008 


Seetion  4.0 
Revision  No. :  1 
Date:  August  2007 
Page  3-11 


URS  converted  the  random  lidar  point  data  to  GIS  produets  using  the  following  steps: 

•  The  lidar  points  were  converted  to  Arcinfo  point  shape  file  format,  with  the 
horizontal  loeations  determined  by  the  northing  and  easting  values  in  the  lidar  point 
file,  and  the  elevation  value,  intensity  value,  and  the  code  for  ground  or  non-ground 
return  retained  as  attributes. 

•  Point  shape  files  were  converted  to  Triangulated  Irregular  Network  (TIN)  files.  A 
TIN  is  an  elevation-based  surface  model  where  each  point  forms  a  vertex  in  a 
network  of  linked  triangles.  TINs  were  created  from  the  lidar  points  coded  as  ground 
returns,  using  Arcinfo ’s  TIN  creation  functions.  A  TIN  was  created  for  each  lidar 
flight  separately.  For  the  Kirtland  data,  an  additional  TIN  was  created  for  the 
combined  300  meter  flights,  the  combined  450  meter  and  900  meter  flights,  and  the 
points  from  all  four  flights.  For  the  Victorville  data,  a  TIN  was  created  containing 
point  from  both  flights.  TINs  were  also  ereated  in  selected  areas  using  both  ground 
and  non-ground  returns,  in  order  to  derive  vegetation  heights  and  to  allow  for  QA/QC 
review  of  point  elassification  as  ground  or  non-ground  returns. 

•  Digital  elevation  models  (DEMs)  were  created  from  the  TINs.  A  DEM  is  a  regularly 
spaeed,  gridded  array  of  elevation  values.  DEMs  were  ereated  using  the  eaeh  of  the 
TIN  files  as  inputs.  DEMs  were  created  in  the  Arcinfo  GRID  file  format,  which 
allows  for  additional  analysis  that  cannot  be  performed  directly  on  the  TIN  file.  All 
DEMs  were  created  using  0.32  meter  (1  foot)  grid  cell  sizes.  This  value  was  ehosen 
as  the  smallest  cell  size  that  would  be  supported  by  the  lidar  data  densities  acquired. 

Automated  GIS  processing  and  analysis  was  used  for  many  of  the  process  steps. 

Once  the  initial  GIS  products  were  created,  the  lidar  data  were  examined  to  detect  missing  data, 
spatial  discrepancies,  or  artifacts  in  the  surface  that  would  indicate  improper  calibration  or  other 
problems.  Eurther  data  quality  review,  including  review  for  spatial  accuracy,  was  performed 
based  on  the  parameters  given  in  Appendix  C  of  the  Technology  Demonstration  Plan.  All  data 
met  data  quality  speeifications.  Detailed  results  are  given  in  Appendix  A,  Eidar  and  Orthophoto 
Positional  Aceuraey  Results. 

hollowing  ereation  of  initial  GIS  produets  and  initial  QA/QC  review,  analysis  of  the  lidar  and 
orthophoto  data  was  conducted  by  URS: 

•  Hillshades  were  created  for  each  of  the  DEMs.  Hillshades  are  three-dimensional 
depletions  of  the  surface  with  shadows  formed  by  a  simulated  light  source  placed 
above  the  surface  at  an  altitude  and  azimuth  chosen  by  the  operator.  The  default 
settings  for  hillshades  in  ArcGIS  Spatial  Analyst  were  used,  then  varied  as  needed 
during  the  analysis.  Hillshades  were  saved  in  Arcinfo  GRID  format. 
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•  Each  lidar  hillshade  and  orthophoto  data  set  was  visually  inspected  for  potential 
MRS.  Potential  MRS  were  identified  and  drawn  as  Arcinfo  line  or  polygon  shape 
files  for  eaeh  data  set. 

•  Eaeh  data  set  was  visually  inspeeted  for  potential  munitions-related  features. 

Potential  features  were  drawn  as  Arcinfo  line  or  polygon  shape  files  for  eaeh  data  set. 
Potential  features  were  classified  aecording  to  naming  standards  provided  by  ESTCP. 

•  A  slope  model  was  ereated  for  eaeh  demonstration  site.  The  proeess  used  the  default 
setting  for  the  AreGIS  SEOPE  GRID  eommand. 

•  Models  of  elevation  above  and  below  an  average  ground  surface  were  ereated  for 
eaeh  lidar  data  set.  The  proeess  used  the  Areinfo  Eoeal  Mean  function. 

3.7.4  Detection  and  Delineation  of  Munitions  Response  Sites 

At  the  Kirtland  site,  all  four  of  the  MRS  deseribed  in  the  Kirtland  CSM  were  detected  using 
lidar,  and  three  of  the  four  were  detected  using  orthophotos.  In  addition,  both  teehnologies 
revealed  additional  potential  sites  of  interest. 

Table  3-2  presents  Kirtland  MRS  deteetion  results  for  each  of  the  lidar  and  orthophoto  data  sets. 


Table  3-2 

MRS  Detection  -  Kirtland 


MRS  from  CSM 

Additional 

Sites  of  Interest 

N2 

N3 

NDIA 

SORT 

Orthophotos 

20  cm 

Y 

Y 

Y 

N 

1 

10  cm 

Y 

Y 

Y 

N 

1 

Individual  lidar  flights 

900  m 

Y 

Y 

Y 

Y 

6 

450  m 

Y 

Y 

Y 

Y 

7 

300  m  flight  1 

Y 

Y 

Y 

Y 

7 

300  m  flight  2 

Y 

Y 

Y 

Y 

7 

Combined  lidar  flights 

450  and  900  m 

Y 

Y 

N 

Y 

6 

Both  300  m 

Y 

Y 

Y 

Y 

5 

All  lidar 

Y 

Y 

N 

Y 

5 

As  lidar  data  from  different  flights  was  combined,  the  number  of  additional  potential  MRS 
identified  and  the  overall  area  mapped  as  MRS  began  to  drop.  This  phenomenon  is  described 
more  fully  in  Appendix  B,  Combining  Lidar  Data  from  Multiple  Elights. 
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At  the  Victorville  site,  both  MRS  described  in  the  initial  CSM  were  detected,  and  were  visible  in 
both  lidar  data  sets  and  the  single  orthophoto  data  set.  No  additional  sites  of  interest  were 
detected  at  Victorville  using  these  technologies. 

The  following  sections  review  data  for  each  MRS  listed  in  the  Kirtland  CSM  in  more  detail. 

3.7.5  Kirtland  -  Target  N3  Results 

Target  N3  is  described  in  the  CSM  as  a  1,000-foot  diameter  bull’s-eye  target  used  for  100-pound 
practice  bombs  and  scrap  storage.  In  addition  to  the  bull’s-eye  rings,  three  potential  ancillary 
targets  and  one  additional  area  of  interest  were  identified  that  were  not  listed  in  the  CSM:  a  ship 
target  east  of  the  bull’s-eye,  a  simulated  airfield  target  to  the  south,  a  diamond-shaped  potential 
target  to  the  west,  and  a  small  rectangular  feature  to  the  north.  The  bull’s-eye  rings  consist  of 
berms  between  5  and  10  cm  high  and  a  mound  at  the  center  point  approximately  80  cm  high. 

The  ship  target  consists  of  berms  ranging  from  60  cm  in  height  on  the  north  end  to  essentially 
flush  with  the  ground  surface  on  the  south  end.  The  diamond-shaped  potential  target  consists  of 
trenches  from  20  to  70  cm  deep  in  the  corners. 


Table  3-3 

Target  N3  Detection  Results 


Rings 

Ship 

Airstrip 

Diamond 

Small 

Rectangle 

Total  Area  of 
MRS 
(m^) 

Orthophotos 

10  cm 

5 

N 

N 

Y 

N 

20  cm 

5 

N 

N 

Y 

N 

195,258 

Individual  lidar 

900  m 

4 

Y 

Y 

Y 

N 

992,977 

450  m 

5 

Y 

Y 

Y 

N 

1,226,982 

300  ml 

5 

Y 

Y 

Y 

Y 

1,226,982 

300  m2 

5 

Y 

Y 

Y 

Y 

1,226,982 

Combined  lidar 

450  and  900  m 

5 

Y 

Y 

Y 

N 

1,226,982 

Both  300  m 

5 

Y 

992,977 

All  lidar 

5 

Y 

Y 

Y 

Y 

992,977 
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Figure  3-10  shows  the  Target  N3  target  elements  deteeted. 


Figure  3-10 
Target  N3  Elements 


Target  N3,  300  m  lidar,  showing  target  bull’s-eye 
rings,  ship  target  (east),  diamond-shaped  potential 
target  (west),  and  simulated  air  strip  (south). 


Diamond-shaped  potential  target  west  of  N3  bull’s-eye, 
20  cm  pixel  orthophoto.  Feature  is  approximately  100 
m  on  each  side. 


Target  N3  bull’s-eye  rings,  450  m  lidar  displayed 
as  elevations  above  an  average  ground  surface. 


Diamond-shaped  potential  target  west  of  N3  bull’s-eye, 
300  m  lidar.  Feature  is  similar  in  all  lidar  data  sets. 
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Figure  3-10  (continued) 
Target  N3  Elements 


Ship  target  east  of  N3,  10  em  pixel  orthophoto. 
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Airstrip  target  south  of  N3  bull’s-eye. 


3,7,6  Kirtland  -  Target  N2  Results 


Ship  target  east  of  N3  shown  in  300  m  lidar  surfaee 
eombined  with  a  300  m  lidar  subtraeted  grid  to 
enhanee  the  image. 


Small  reetangular  area  of  interest  north  of  N3 
bull’s-eye. 


Target  N2  is  described  in  the  CSM  as  a  bull’s-eye  target  used  for  100-pound  practice  bombs. 
The  bull’s-eye  rings  were  visible  using  lidar  data;  there  were  few  to  no  visible  craters.  Three 
rings  were  visible  using  the  orthophotos  at  both  pixel  sizes,  and  four  were  visible  in  the  lidar 
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data,  regardless  of  data  density.  Because  of  the  different  number  of  rings  visible,  the  total  area  of 
this  MRS  from  the  orthophotos  was  26,234  m  ,  while  the  total  area  from  the  lidar  data  was 
74,753  m^.  The  bull’s-eye  rings  consist  of  berms  between  5  and  10  cm  high.  Figure  3-11  shows 
the  Target  N2  target  elements  detected. 


Figure  3-11 

Target  N2  Orthophoto  and  Lidar 


Target  N2  bull’s-eye,  20  em  pixel  orthophoto  Target  N2  bull’s-eye,  300  m  lidar 

3,7,7  Kirtland  -  Target  NDIA  Results 

The  NDIA  is  described  in  the  CSM  as  a  1,000  foot  diameter  HE  bulTs-eye  target.  Target 
cross-hairs  and  a  single  bull’s-eye  ring  were  visible  in  the  orthophotos  at  both  pixel  sizes;  neither 
was  visible  in  the  lidar  data  at  any  data  density.  A  large  number  of  probable  craters  was  visible 
in  the  lidar  data,  with  the  number  counted  increasing  with  data  density,  then  decreasing  as  data 
from  the  individual  flights  was  combined.  These  craters  were  used  to  define  the  boundary  of  the 
MRS.  Craters  were  not  visible  in  the  orthophoto  data.  Table  3-4  shows  the  features  detected 
with  each  data  set.  Potential  craters  ranged  from  1  to  3  meters  in  diameter  and  from  10  to  20  cm 
in  depth. 
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Table  3-4 

Target  NDIA  Detection  Results 


Prohahle 

Craters 

Rings 

Cross-hairs 

Total  Area 
(m^) 

Orthophotos 

10  em 

0 

1 

2 

78,251 

20  em 

0 

1 

2 

78,251 

Individual  Udar 

900  m 

25 

0 

0 

182,356 

450  m 

34 

0 

0 

182,356 

300  m  1 

44 

0 

0 

182,356 

300  m2 

39 

0 

0 

182,356 

Combined  lidar 

450  and  900  m 

34 

0 

0 

not  found 

Both  300  m 

29 

0 

0 

182,356 

All  lidar 

33 

0 

0 

not  found 

Figure  3-12  illustrates  the  eomplementary  ability  of  lidar  and  orthophotos  to  define  this  target 
area,  with  the  orthophoto  image  showing  the  cross-hairs  and  target  rings,  and  the  lidar  data 
showing  the  probable  craters. 


Figure  3-12 

Target  NDIA  Orthophoto  and  Lidar 


Target  NDIA  20  em  orthophoto  showing  target 
erosshairs  and  bull’s-eye  ring.  Craters  eould  not  be 
seen  in  the  orthophotos. 


Target  NDIA  300  m  lidar  showing  eraters.  Cross¬ 
hairs  and  bull’s-eye  ring  eould  not  be  seen  using 
lidar. 
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3,7,8  Kirtland  -  Target  SORT  Results 

The  SORT  is  described  in  the  CSM  as  a  suspected  target  whose  location  is  given  as  “somewhere 
northwest  of  the  airport.”  The  target  was  identified  as  a  23-acre  rectangle  subdivided  into  cells 
simulating  an  oil  tank  farm.  This  target  was  located  using  lidar  data  at  all  densities,  but  was  not 
visible  in  either  of  the  orthophoto  sets  (Figure  3-13).  The  target  cells  consist  of  berms  from  0  to 
20  cm  in  height. 


Figure  3-13 

Target  SORT  Orthophoto 


Target  SORT,  10  cm  pixel  orthophotos.  The  target  was  Target  SORT,  300  m  lidar.  The  target  was  visible  in 
not  visible  in  either  orthophoto  data  set.  lidar  data  at  all  densities  collected. 

3,7,9  Kirtland  -  Other  Sites  of  Interest 

At  the  Kirtland  site,  15  additional  sites  of  interest  were  detected  using  the  lidar  data,  including 
seven  in  the  north  portion  of  the  site  and  eight  in  the  south  portion  (Figure  3-14).  One  area  in  the 
south  portion  consisted  of  a  previously  unknown  potential  target  bull’s-eye;  the  remainder 
consisted  of  small  groupings  of  potential  craters  and  the  small  rectangular  feature  north  of  Target 
N3.  All  of  these  potential  MRS  were  detected  using  lidar  data;  none  was  visible  in  the 
orthophotos.  The  potential  craters  are  generally  larger  and  more  irregular  than  would  be 
expected  from  any  of  the  munitions  mentioned  in  the  initial  CSM,  and  their  sizes  are  irregular.  It 
is  possible  that  these  features  may  be  related  to  burial  or  other  munitions  management  activities; 
however,  the  lidar  data  do  not  indicate  a  high  confidence  that  these  areas  are  related  to  munitions 
use.  Rather,  these  are  areas  that  warrant  further  investigation. 
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Figure  3-14 

Sample  Additional  Sites  of  Interest 


Previously  unknown  potential  target  bull’s-eye,  10  em 
pixel  orthophoto.  Potential  target  was  not  visible. 
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Craters  forming  potential  MRS,  450  m  lidar.  This  area 
is  identified  in  the  CSM  as  KPBR-AOI-10. 


Previously  unknown  potential  target  bull’s-eye, 
south  portion  of  site,  300  m  lidar.  Identified  in 


CSM  as  KPBR-AOI-4. 
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Craters  forming  potential  MRS,  elevations  eoded  as 
below  (red)  and  above  (green)  the  ground  surfaee. 


3,7,10  Victorville  -  Target  DBT  Y  Results 

Target  DBT  Y  is  described  in  the  CSM  as  the  bed  of  Means  Dry  Lake,  an  area  of  approximately 
345  acres.  This  area  was  clearly  visible  using  both  lidar  and  orthophotos,  and  the  delineated 
boundary  of  the  dry  lake  bed  was  not  different  between  the  lidar  and  orthophotos,  nor  for  any  of 
the  lidar  data  densities  tested  (Figure  3-15). 
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Figure  3-15 

Target  DBT  Y  Orthophoto  and  Lidar 


Target  DBT  Y,  orthophoto  Target  DBT  Y,  eombined  lidar 


3,7,11  Victorville  -  Target  PBR  15  Results 

Target  PBR  15  is  described  in  the  CSM  as  a  bull’s-eye  target  used  for  precision  bombing 
practice.  The  target  was  clearly  visible  in  the  orthophotos.  Initially,  the  target  was  difficult  to 
detect  using  lidar,  since  the  bull’s-eye  rings  are  formed  from  asphalt  laid  flat  on  the  relatively  flat 
ground  surface.  The  target  thus  has  very  little  vertical  relief  to  be  detected  by  lidar.  However, 
an  image  of  the  area  created  using  the  lidar  intensity  values  showed  the  target  clearly 
(Figure  3-16).  Target  ring  dimensions  were  60  m  (197  feet)  for  the  smallest  ring,  120  m 
(394  feet)  for  the  middle  ring,  and  190  m  (623  feet)  for  the  outer  ring. 
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Figure  3-16 

Target  PBR  15  Orthophotos,  Lider  Surface  Model  and  Lidar  Intensity  Image 


Target  PBR  15,  450  m  lidar. 


Target  PBR  15,  300  m  lidar. 


Target  PBR  15,  lidar  intensity. 


3,7,12  Detection  of  Potential  Munitions-Related  Ground  Features 


The  Kirtland  site  eontained  just  over  one  hundred  small  features  that  were  potentially  related  to 
munitions  use,  mostly  potential  craters.  The  site  also  contained  numerous  linear  features,  mostly 
jeep  trails  and  other  vehicle  tracks.  Potential  features  were  visually  identified  from  the  lidar 
hillshades  and  the  orthophotos.  The  extremely  shallow  nature  of  the  potential  craters  near  Target 
NDIA  (10-20  cm)  made  many  of  them  difficult  to  distinguish  with  confidence  from  the  natural 
ground  surface.  Consequently,  the  numbers  given  for  objects  detected  should  be  understood  as  a 
comparison  of  the  various  data  sets  rather  than  a  conclusive  number  of  MEC-related  features. 
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The  number  of  features  detected  at  the  Kirtland  site  also  varied  depending  on  the  method  used  to 
display  the  lidar  data  (Figure  3-17),  particularly  for  the  weathered  craters  near  Target  NDIA. 
This  suggests  that  these  potential  craters  were  near  the  detection  limit  of  this  technology. 


Figure  3-17 

Kirtland  Target  NDIA  Potential  Crater  Detection 


center,  10  cm  pixel  orthophoto. 


;  . 
I 


Target  NDIA,  craters  identified 
from  lidar  hillshade  alone. 


Target  NDIA,  craters  identified 
from  lidar  surface  model  color- 
coded  to  show  depths  of 
depressions. 


At  Victorville,  potential  craters  in  Means  Dry  Lake  were  much  larger  than  at  any  of  the  Kirtland 
targets,  ranging  from  5  to  8  m  in  diameter  and  up  to  1  m  deep.  Unlike  the  smaller  craters  at 
Kirtland,  the  craters  at  Victorville  were  clearly  visible  in  the  orthophotos  (Figure  3-18).  A 
significant  number  of  potential  craters  were  located  outside  and  northwest  of  the  lake  boundary. 


Figure  3-18 

Victorville  Target  DBT/Craters,  Orthophotos  and  Lidar 


Victorville  Target 
surface  model  including  vegetation  point 


FINAL  REPORT 

High  Density  Lidar  and  Orthophotography 
in  UXO  Wide  Area  Assessment 
Contraet  No.  N44255-02-D-2008 


Seetion  4.0 
Revision  No.:  1 
Date:  August  2007 
Page  3-23 


Figure  3-18 

Victorville  Target  DBT/Craters,  Orthophotos  and  Lidar  (continued) 


Vietorville  Target  DBT  Y  eraters,  lidar  surfaee 
model,  ground  points  only 


Vietorville  Target  DBT  Y  eraters,  lidar  surface 
model,  ground  points  only,  color-coded  to 
show  feature  heights  and  depths. 


The  potential  crater  locations  outside  of  the  dry  lake  bed  were  used  to  modify  the  location  of  the 
target  area  boundary.  Following  this  modification,  the  size  of  the  potential  MRS  increased  from 
the  original  345  acres  to  641  acres  (Figure  3-19). 


Figure  3-19 

Victorville  Target  DBT  Y  -  MRS  Boundaries 


Victorville  Target  DBT  Y,  MRS  boundary 
from  CSM. 


Victorville  Target  DBT  Y,  MRS  boundary 
following  modification  using  feature  locations. 


FINAL  REPORT 

High  Density  Lidar  and  Orthophotography 
in  UXO  Wide  Area  Assessment 
Contraet  No.  N44255-02-D-2008 


Seetion  4.0 
Revision  No.:  1 
Date:  August  2007 
Page  3-24 


No  craters  were  detected  at  Target  PBR  15.  This  is  consistent  with  its  reported  use  as  a  practice 
bombing  target.  There  were  no  other  potential  munitions-related  features  at  the  target,  other  than 
the  asphalt  target  rings  themselves. 

3,7,13  Data  Density  Effects  -  General  Results 

The  test  crater  area  at  the  Kirtland  site  was  used  to  examine  the  general  effects  of  lidar  and 
orthophoto  data  density.  Orthophoto  data  density  was  examined  first,  based  on  the  size  of  the 
image  pixels.  Figure  3-20  shows  the  test  crater  area  in  the  10-cm  and  20-cm  pixel  data  sets.  The 
10  cm  orthophotos  and  the  known  dimensions  of  the  test  craters  were  used  to  create  the  “best” 
locations  for  the  test  craters;  these  were  then  superimposed  on  the  20  cm  photo. 


Figure  3-20 

Kirtland  Test  Craters,  Orthophotos 


10  cm  orthophoto  with  “best”  crater  locations. 


20  cm  orthophoto  with  “best”  crater  locations. 


These  images  illustrate  the  substantial  difference  in  clarity  between  the  10  cm  and  20  cm  pixel 
orthophotos.  All  10  test  craters  are  visible  in  the  10  cm  orthophoto,  although  the  0.32  m  craters 
are  distinguishable  from  surrounding  objects  primarily  through  their  regular  pattern.  The  1.5  and 
1.0  m  craters  are  visible  in  the  20  cm  pixel  orthophoto;  however,  neither  can  be  clearly 
distinguished  from  the  surrounding  ground  features.  This  comparison  led  to  the  decision  to 
acquire  only  10  cm  pixel  orthophotos  at  the  Victorville  site. 


Lidar  data  density  was  examined  based  on  the  average  number  of  points  per  square  meter.  Lidar 
data  was  obtained  at  the  overall  data  densities  shown  in  Table  3-5. 
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Table  3-5 

Achieved  Lidar  Data  Densities 


Demonstration  Site 

Flight 

Point  density  (pts/m^) 

Kirtland 

900  m  North  Block 

1.4 

900  m  South  Block 

1.6 

450  m  north  block 

5.2 

450  m  south  block 

4.1 

300  m  east-west  north  block 

5.2 

300  m  north-south  north  block 

6.5 

300  m  flight  1  south  block 

5.9 

300  m  flight  2  south  block 

6.1 

Victorville 

450  m  flight 

4.8 

300  m  flight 

6.4 

Combined  300  and  450  m  flights 

11.2 

These  overall  lidar  data  densities  are  a  general  representation  based  on  statisties  for  the  site  as  a 
whole.  In  faet,  lidar  data  density  varies  eonsiderably  over  the  ground  surfaee,  a  eomplex 
phenomenon  that  is  diseussed  in  Appendix  C. 

Figure  3-21  shows  hillshades  of  the  four  lidar  data  sets  at  the  Kirtland  test  eraters,  using  the 
ground  returns  only.  The  loeations  of  the  test  eraters  are  taken  from  the  10  em  orthophoto  data 
and  superimposed  on  the  lidar  images. 


Figure  3-21 

Kirtland  Test  Craters,  Lidar  Surface  Models 


Kirtland  test  craters,  900  m  (1.6  pts/  m^)  lidar 
hillshade,  ground  points  only. 


Kirtland  test  craters,  450  m  (4.1  pts/  m^)  lidar 
hillshade,  ground  points  only 
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Figure  3-21 

Kirtland  Test  Craters,  Lidar  Surface  Models  (continued) 


Kirtland  test  eraters,  300  m  (5.9  pts/  m^)  lidar  Kirtland  test  eraters,  300  m  (6. 1  pts/  m^)  lidar 

flight  1  hillshade,  ground  points  only.  flight  2  hillshade,  ground  points  only. 


These  images  show  that  eraters  are  more  elearly  defined  as  the  density  of  the  lidar  points 
inereases.  In  the  900  m  lidar  data,  the  erater  area  appears  as  a  single  depression  and  the 
individual  eraters  eannot  be  deteeted.  In  the  450  m  lidar  data,  the  1.5  and  1.0  meter  eraters  ean 
be  distinguished,  but  the  0.32  meter  eraters  are  not  visible.  In  the  300  m  lidar  data,  the  1.5  and 
1.0  meter  diameter  eraters  are  better  defined,  and  the  group  of  0.32  meter  eraters  begins  to  be 
seen  as  a  single  depression.  None  of  the  lidar  data  sets  showed  the  0.32  meter  eraters 
individually. 

At  the  Vietorville  site,  only  one  orthophoto  data  set  was  aequired.  For  the  lidar  data,  the  data 
from  the  300  m  and  450  m  lidar  flights  were  sueeessfully  aligned  to  ereate  a  eombined  data  set, 
yielding  a  total  of  three  lidar  data  densities  (Figure  3-22).  As  at  Kirtland,  the  data  showed  that 
feature  definition  inereased  with  inereasing  lidar  density. 
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Figure  3-22 

Victorville  Test  Craters,  Lidar  Surface  Models 


These  images  support  the  findings  at  the  Kirtland  site  that  increased  lidar  data  density  leads  to 
better  detection  of  smaller  features.  The  Victorville  test  craters  are  somewhat  harder  to 
distinguish  from  the  surrounding  area  because  they  are  not  grouped  together.  At  the  lower  data 
density  of  the  450  m  flight,  the  1.0  and  1.5  m  craters  are  difficult  to  distinguish,  while  both  are 
clearly  visible  in  the  300  m  data  and  the  combined  data.  The  0.32  m  crater  is  discemable  in  the 
combined  data  set;  however,  without  a  group  of  craters  in  a  regular  pattern  such  as  at  Kirtland,  it 
would  be  difficult  to  distinguish  from  the  surrounding  natural  variation  in  the  ground  surface. 
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3,7,14  Data  Density  Effects  -  Quantitative  Results 

Counts  were  made  of  potential  features  identified  in  each  lidar  and  orthophoto  data  set.  For  the 
Kirtland  site,  two  classes  of  features  were  found;  line  features  such  as  roads,  trails  and  the  target 
bull’s-eye  rings,  and  area  features  such  as  probable  craters,  isolated  depressions  that  did  not 
appear  to  be  craters,  and  other  items  of  interest  including  regularly-shaped  mounds 
(Figure  3-23).  The  orthophotos  area  features  included  what  appeared  to  be  old  building 
foundations  and  articles  of  debris  on  the  surface.  Table  3-6  presents  results  for  each  data  set. 


Figure  3-23 

Kirtland  Sample  Features  of  Interest 


Debris  on  the  ground,  approximately  1.0  x  0.5  m  Unidentified  depression,  approximately  2  m  in 


diameter 

Table  3-6  summarizes  the  number  of  features  identified  for  each  data  set  at  the  Kirtland  site. 
Figures  3-24  and  2-25  show  the  number  of  area  features  and  the  overall  length  of  linear  features 
detected  for  each  orthophoto  and  lidar  data  set. 

Area  features  included  depressions  that  did  not  appear  to  be  craters,  probable  craters,  and  other 
features  of  interest.  Linear  features  were  primarily  dirt  roads  and  jeep  trails.  The  relatively 
small  number  of  roads  and  trails  at  the  Kirtland  site,  along  with  their  faint  and  eroded  character, 
make  them  appropriate  subjects  for  comparison  of  the  different  data  sets.  Total  length  of 
features  was  used  in  place  of  the  number  of  features  to  avoid  problems  with  potentially  arbitrary 
division  of  the  roads  and  trails  into  segments. 
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Table  3-6 

Kirtland  Potential  Features 


Area  Features 

Line  Features 

Depressions 

Prohahle 

Craters 

Other 
Features  of 
Interest 

Total  Area 
Features 

Total  Length  of 
Line  Features  (m) 

Orthophotos 

20  em 

0 

0 

29 

29 

92,325 

10  em 

0 

0 

49 

49 

83,658 

Individual  lidar 

900  m 

23 

31 

16 

70 

78,047 

450  m 

30 

48 

20 

98 

89,907 

300  m  1 

52 

63 

23 

138 

97,968 

300  m2 

56 

61 

29 

146 

98,220 

Combined  Udar 

450  and  900  m 

36 

44 

23 

103 

89,948 

Both  300  m 

51 

49 

29 

129 

89,974 

All  lidar 

44 

47 

29 

120 

84,588 

Figure  3-24 

Kirtland  Area  Features  by  Data  Source 


1  2  900  m  300  m  Lidar 

Data  Source 
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Figure  3-25 

Kirtland  Line  Feature  Length  by  Data  Source 


900  m  300  m 

Data  Source 


The  data  show  that  the  number  of  area  features  such  as  potential  craters  was  higher  for  lidar  than 
for  orthophotos,  and  the  number  of  features  identified  increased  with  data  density  for  both  lidar 
and  orthophotos.  However,  when  lidar  data  from  different  altitudes  was  combined,  the  number 
of  features  detected  decreased.  Line  features  were  detected  roughly  equally  using  either 
technology. 

At  the  Victorville  site,  results  were  more  equal  between  the  two  technologies,  probably  due  to 
the  larger  size  of  the  craters  (Table  3-7).  A  slightly  smaller  number  of  craters  was  counted  using 
the  orthophotos,  probably  as  a  result  of  the  vegetation  in  some  parts  of  the  Means  Dry  Lake  bed 
that  obscured  the  craters  in  the  orthophotos  but  not  the  lidar  data.  A  slightly  higher  number  of 
additional  features  were  identified  using  both  the  orthophotos  and  lidar  together. 


Table  3-7 

Victorville  Potential  Features 


Total  Features 

Prohahle  Craters 

Orthophoto 

10  cm 

1171 

1108 

Lidar 

450  m 

1185 

1108 

300  m 

1183 

1106 

300  and  450  m 

1186 

1105 

Orthophoto  and  lidar 

1198 

1121 
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Figure  3-26  shows  the  number  of  area  features  for  eaeh  orthophoto  and  lidar  data  set  at  the 
Victorville  site. 


Figure  3-26 

Victorville  Features  by  Data  Source 


0  10  cm  450  m  300  m  300  and  450  Orthophoto 

m  and  lidar 


Data  Source 


Line  features  were  not  quantitatively  analyzed  at  Victorville,  as  the  site  is  covered  with  off-road 
vehicle  tracks  at  a  density  that  made  quantitative  analysis  difficult.  However,  roads  and  off-road 
vehicle  tracks  are  clearly  visible  in  both  the  lidar  and  orthophoto  data  at  all  data  densities 
obtained. 

3,7,15  Flight  Line  Orientation  Effects 

At  the  Kirtland  site,  the  two  300  m  lidar  flights  were  flown  perpendicular  to  each  other  in  the 
north  portion  of  the  site  (Figure  3-27).  The  resulting  surface  models  were  compared  to 
determine  whether  flight  line  orientation  would  affect  feature  detection.  The  data  showed  that 
faint  jeep  roads  running  east- west  were  more  clearly  visible  when  the  lidar  flight  lines  are  also 
east-west.  This  effect  appears  to  be  due  to  the  orientation  of  the  sweep  of  the  laser  either  along 
or  across  these  shallow  roads.  More  distinct  features,  such  as  the  larger  north-south  road  in  the 
images  below,  were  equally  visible  in  both  data  sets. 


FINAL  REPORT 

High  Density  Lidar  and  Orthophotography 
in  UXO  Wide  Area  Assessment 
Contraet  No.  N44255-02-D-2008 


Seetion  4.0 
Revision  No.:  1 
Date:  August  2007 
Page  3-32 


Figure  3-27 

Kirtland  Flightline  Orientation  Effects 


Roads,  300  m  lidar  east-west  flight  lines. 
Arrow  shows  flight  line  direetion.  East-west 
roads  appear  more  elearly. 


Roads,  300  m  lidar.  North-south  flight  lines. 
Arrow  shows  flight  line  direetion.  East-west 
roads  appear  less  elearly. 


3,7,16  Data  Artifacts  and  Noise  Effects 


Two  types  of  data  artifacts  were  encountered  in  the  lidar  data.  First,  where  data  from  two 
parallel  flight  lines  overlapped,  the  surface  occasionally  showed  very  shallow  (roughly  0.05  m) 
meandering  linear  features.  This  is  a  well-known  type  of  noise  effect  caused  by  slight 
discrepancies  between  data  between  the  two  flight  lines.  The  linear  feature  appears  at  the 
boundary  of  the  overlap  of  the  flight  line  overlap  area  (Figure  3-28).  Such  anomalies  are 
probably  the  result  of  small  errors  in  the  GPS,  IMU,  and  laser  range  finder  that  cannot  be 
adjusted  out  during  data  processing. 


FINAL  REPORT 

High  Density  Lidar  and  Orthophotography 
in  UXO  Wide  Area  Assessment 
Contraet  No.  N44255-02-D-2008 


Seetion  4.0 
Revision  No.:  1 
Date:  August  2007 
Page  3-33 


Figure  3-28 

Kirkland  Flight  Line  Data  Overlap  Effect 


Flight  line  artifaet,  300  m  lidar.  The  faint  Flight  line  artifaet,  bold  red  lines  represent 

linear  feature  shown  does  not  appear  in  lidar  flight  lines, 

surfaees  with  different  flight  line 
orientations. 

Second,  the  lidar-derived  ground  surface  often  showed  a  “corduroy”  effect  consisting  of  shallow 
(roughly  0.05  m)  ridges  running  perpendicular  to  the  flight  line  and  along  the  sweep  of  the  laser 
(Figure  3-29).  This  is  also  a  well-known  lidar  data  artifact.  This  effect  also  likely  results  from 
small  errors  in  the  GPS,  IMU  and  laser  range  finder.  The  size  of  both  anomalies  is  well  within 
the  vertical  accuracy  specifications  for  the  data. 
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Figure  3-29 

Victorville  Lidar  “Corduroy”  Effect 


3.7.17  Vegetation  Patterns 

Vegetation  at  the  Kirtland  site  was  examined  in  both  orthophotos  and  lidar  data  to  determine 
whether  vegetation  patterns  eould  reveal  surface  disturbances  related  to  munitions  use  (Figure  3- 
30).  Orthophotos  were  examined  visually,  and  for  the  lidar  data,  vegetation  heights  were 
modeled.  The  bombing  targets  were  occasionally  very  slightly  discemable  in  the  lidar  vegetation 
height  models;  however,  it  is  unlikely  that  vegetation  data  alone  could  be  used  to  locate  MRS  or 
munitions-related  features. 
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Figure  3-30 

Kirtland  Vegetation  Patterns 


Target  N3,  diamond-shaped  target  to  the  west  Target  N3,  diamond-shaped  target  to  the  west  of 

of  main  target  area,  20  em  orthophoto.  main  target  area,  lidar  vegetation  height  model. 


The  radiating  bars  to  the  west  are  power  lines. 


Target  N2,  bull’s-eye,  lidar  vegetation  height 
model. 


3,7,18  Lidar  and  Orthophoto  Positional  Accuracy 

Understanding  the  positional  accuracy  of  lidar  and  orthophoto  data  is  important  since  both  data 
sets  will  be  integrated  with  a  wide  variety  of  other  spatial  data.  Positional  accuracy 
specifications  were  established  in  the  Demonstration  Plan  for  each  site,  and  are  shown  in  Table 
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4-1  in  Section  4.  Positional  accuracy  verification  methods  are  discussed  in  detail  in  Appendix  A, 
Lidar  and  Orthophoto  Positional  Accuracy  Results. 

Lidar  and  orthophoto  data  met  the  positional  accuracy  criteria  at  both  sites.  Table  3-8  presents 
the  overall  positional  accuracy  results  for  the  two  sites. 


Table  3-8 

Overall  Positional  Accuracy  Results 


Item 

Performance  Criteria 

Method 

Results  (m) 

Lidar  vertical 
accuracy 

Ave.  dz:  -1-/-  0. 1 5  m  compared  to 
control  points 

Kirtland:  lidar  to  ESTCP  control  points 

Ave.  dz:  0.1 1 1 

Kirtland:  lidar  to  TRSI  control  points 

Ave.  dz:  0.088 

Victorville:  lidar  to  TRSI  control  points 

Ave.  dz:  0.102 

Lidar  horizontal 
accuracy 

Ave.  dx/y:  +!-  0.65  m  compared 
to  control  points 

Kirtland:  Average  x  and  y  displacement 
(dx  and  dy)  for  all  control  points  and  all 
lidar  flights. 

Ave.  dx:  0.080 

Ave.  dy:  0.080 

Victorville:  Average  x  and  y 
displacement  (dx  and  dy)  for  all  control 
points  for  each  flight. 

300  m  flight 

Ave.  dx:  0.030 

Ave.  dy:  0.080 

450  m  flight 

Ave.  dx:  0.060 

Ave.  dy:  0.030 

Orthophoto 
horizontal  accuracy 

Ave.  dx/y  under  3  pixel  widths 
compared  to  control  points 

Kirtland  20  cm  orthophotos  to  control 
points 

Ave.  dx:  0.128 

Ave.  dy:  0.139 

Kirtland  10  cm  orthophotos  to  control 
points 

Ave.  dx:  0.077 

Ave.  dy:  0.106 

Victorville  10  cm  orthophotos  to  control 
points 

Ave.  dx:  0.027 

Ave.  dy:  0.037 

Orthophoto  to  lidar 
alignment 

Ave.  dx/dz  under  2  pixel  widths 

Kirtland  20  cm  orthophotos  to  lidar 
positions 

Ave.  dx:  0.167 

Ave.  dy:  0.360 

Kirtland  10  cm  orthophotos  to  lidar 
positions 

Ave.  dx:  0.137 

Ave.  dy:  0.167 

Victorville  10  cm  orthophotos  to  lidar 
positions 

Ave.  dx:  -0.10 

Ave.  dy:  -0.10 
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3,7,19  Other  Observations 

Orthophotos  were  espeeially  useful  for  providing  additional  information  about  potential  surface 
features.  In  Figure  3-31,  orthophotos  were  able  to  clarify  a  feature  that  appeared  to  have  moved 
between  two  lidar  flights.  Orthophotos  showed  that  the  feature  was  related  to  construction  and 
had  in  fact  moved. 


Figure  3-31 

Use  of  Orthophotos  to  Resolve  Apparent  Lidar  Data  Discrepancy 


Unidentified  feature,  20  em  ortho  taken  with 
the  900  m  lidar  flight,  the  orthophoto  reveals 
that  the  feature  is  related  to  ongoing 
eonstruetion  aetivity. 


following  day.  The  feature  appears  to  have 
moved. 


Unidentified  feature,  10  em  ortho  taken  with 
the  450  m  lidar  flight. 
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4,0  PERFORMANCE  ASSESSMENT 

4,1  PERFORMANCE  CRITERIA 

Performance  criteria  for  this  demonstration  were  established  in  the  Demonstration  Plan  for  each 
site,  and  are  shown  in  Table  4-1.  The  lidar  and  orthophoto  data  met  these  performance  criteria, 
as  shown  in  Tables  4-2  and  4-3.  The  results  of  validation  activities  and  the  implications  of  the 
validation  results  discussed  in  Section  4.2. 


Table  4-1 

Performance  Criteria 


Performance  Criteria 

Description 

Primary  or 
Secondary 

Pre-mobilization 

Verification  of  survey 
control  point  positions 

Verify  survey  control  point  locations  within  at  least  3rd  order 
accuracy 

Primary 

Lidar  data  collection  and  processing 

Area  coverage 

100%  coverage  for  each  flight 

Primary 

Lidar  point  density  - 
Kirtland 

Achieve  overall  lidar  point  densities  of: 

200  m  flights  (2)  -  8  pts/m^  each 

450  m  flight  (1)  -  3  pts/  m^ 

900  m  flight  (1)  -  2  pts/  m^ 

Primary 

Lidar  point  density  - 
Victorville 

Achieve  overall  lidar  point  densities  of 

300  m  flight  (1)  -  3  pts/  m^ 

450  m  flight  (1)  -  5  pts/  m^ 

Primary 

Lidar  vertical  accuracy 

Vertical  accuracy  of  +/-  15  cm  compared  to  ground  survey 

Primary 

Lidar  horizontal  accuracy 

Horizontal  accuracy  of  +/-  65  cm  compared  to  ground  survey 

Primary 

Orthophoto  data  collection  and  processing 

Orthophoto  area  coverage 

100%  coverage  for  each  flight 

Primary 

Orthophoto  flight  altitude  / 
pixel  size 

450  m  (for  10  cm  pixel  flight)  -  Kirtland  and  Victorville 

900  m  (for  20  cm  pixel  flight) 

Primary 

Orthophoto  horizontal 
alignment  to  Lidar 

Lidar  and  orthophotos  aligned  so  that  target  features  are  not 
displaced  in  the  two  data  sets 

Primary 

Orthophoto  horizontal 
alignment  to  survey  control 
points 

Orthophotos  aligned  to  survey  control  points  so  that  target 
features  are  not  displaced 

Primary 

Munitions  Response  Site  identiUcation  and  analysis 

MRS  identification 

Correctly  identify  all  previously  identified  MRS 

Primary 

MRS  false  alarm  rate 

No  areas  incorrectly  identified  as  MRS 

Primary 

MRS  boundary  delineation 

Correctly  locate  MRS  boundaries  to  +/-  15%  of  ground-truthed 
area 

Primary 

MRS  feature  identification 

Identify  features  presenting  as  munitions-related 
(anthropogenic) 

Primary 
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Table  4-2 

Data  Quality  Metrics — MRS  Identification  and  Analysis 


1 

Analytical 

Objective 

Metric 

Action  to 

Achieve  Metric 

Sampling 
Freqnency  or 
Timing 

Desired 

Resnlt 

Actnal  Resnlt 

MRS  Identification  and  Analysis 

20 

MRS 

identification 

Correctly  identify  all 
previously  identified 
MRS. 

Identify  and  document  MRS 
from  lidar  and  orthophoto 
data  sets. 

Each  lidar  and 
orthophoto  data 
set  and 
combinations. 

Correctly  identify 
all  MRS. 

Kirtland 

Lidar:  Accomplished.  All 
previously  identified  MRS  were 
correctly  identified. 

Orthophotos:  Not  Accomplished. 
One  of  four  MRS  correctly 
identified. 

Victorville 

Lidar:  Accomplished.  Both 
previously  identified  MRS  were 
correctly  identified. 

Orthophotos:  Accomplished. 

Both  MRS  were  correctly 
identified. 

21 

MRS  false 
alarm  rate 

No  areas  incorrectly 
identified  as  MRS. 

Identify  and  document  MRS 
from  lidar  and  orthophoto 
data  sets. 

Each  lidar  and 
orthophoto  data 
set  and 
combinations. 

No  false  MRS 
identification. 

Kirtland 

Not  Accomplished.  15  potential 
MRS  were  identified  that  appear 
not  to  be  munitions-related. 

Victorville 

Accomplished.  No  additional 
potential  MRS  were  identified 
from  the  lidar  or  orthophoto  data. 
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Table  4-2 

Data  Quality  Metrics — MRS  Identification  and  Analysis  (continued) 


1 

Analytical 

Objective 

Metric 

Action  to 

Achieve  Metric 

Sampling 
Freqnency  or 
Timing 

Desired 

Result 

Actual  Result 

22 

MRS 

boundary 

delineation 

Correctly  locate  MRS 
boundaries  to  +/-  15% 
of  ground-truthed  area. 

Identify  and  document  MRS 
boundaries  from  lidar  and 
orthophoto  data  sets  for  a 
selected  set  of  test  MRS. 

Each  lidar  and 
orthophoto  data 
set  and 
combinations. 

Locate  MRS 
boundaries  within 
metrics. 

Kirtland 

Partially  Accomplished.  For 

most  targets,  the  lidar  data  formed 
the  boundary  of  the  MRS. 

However,  at  Targets  N3  and 

SORT,  additional  areas  were 
identified  using  mag  and  EM  that 
changed  the  boundaries  more  than 
15%. 

Victorville 

Partially  Accomplished.  For 

Target  DBT  Y,  validation 
activities  did  not  change  the  MRS 
boundary  established  using  lidar. 
For  Target  PBR  15,  mag  data 
showed  metal  frag  more 
accurately  than  the  boundary 
established  using  lidar. 

23 

MRS  feature 
identification 

Identify  features 
presenting  as  human- 
made  (anthropogenic) 
not  including  craters 
(e.g.,  walls,  berms,  pits, 
small  buildings). 

Lidar  and  photo  data  sets  will 
be  examined  for  linear 
features. 

Each  lidar  and 
orthophoto  data 
set  and 
combinations. 

90%  of  features 
identified  from 
selected  field- 
identified  features. 

Accomplished.  At  both  sites, 
potential  human-made  features 
were  readily  identifiable. 
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Table  4-2 

Data  Quality  Metrics — MRS  Identification  and  Analysis  (continued) 


1 

Analytical 

Objective 

Metric 

Action  to 

Achieve  Metric 

Sampling 
Freqnency  or 
Timing 

Desired 

Result 

Actual  Result 

24 

MRS  feature 
identification 

Identify  craters.  Count 
90%  of  craters  over  1.5 
m  diameter  and  .3  m 
depth. 

Automated  algorithms  will  be 
used  to  identify  and  count 
craters  using  lidar  data. 

Each  lidar  and 
orthophoto  data 
set  and 

combinations. 

90%  of  craters 
identified  outside  of 
crater  fields.  95% 
of  craters  identified 
inside  crater  fields. 

Kirtland 

Unknown.  Validation  activities 
appear  to  be  no  more  accurate 
than  lidar  and  orthophotos  in 
identifying  the  highly  eroded 
craters  at  the  site. 

Victorville 

Unknown.  Validation  activities 
do  not  appear  to  have  been 
conducted  to  count  individual 
craters.  However,  no 
concentrations  of  craters  were 
reported  in  addition  to  those 
detected  using  lidar  and 
orthophotos. 

25 

MRS  feature 
identification 

Identify  vegetation 
patterns  indicating 
previous  disturbance. 

Automated  algorithms  will  be 
used  to  map  vegetation 
heights  and  patterns  from 
lidar  data.  Results  will  be 
examined  for  linearity  or 
other  regular  shapes. 
Orthophotos  will  be 
examined  for  regular 
vegetation  patterns. 

Each  lidar  and 
orthophoto  data 
set  and 
combinations. 

Identification  of 
MRS. 

Not  Accomplished.  Neither 
orthophotos  nor  lidar  revealed 
patterns  of  vegetation  indicating 
previous  disturbance. 
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Table  4-2 

Data  Quality  Metrics — MRS  Identification  and  Analysis  (continued) 


1 

Analytical 

Objective 

Metric 

Action  to 

Achieve  Metric 

Sampling 
Freqnency  or 
Timing 

Desired 

Result 

Actual  Result 

26 

MRS  feature 
identification 

Identify  established 
roads. 

Lidar  and  photo  data  sets  will 
be  examined  for  linear 
features. 

Each  lidar  and 
orthophoto  data 
set  and 
combinations. 

100%  identification 
of  established 
roads. 

Accomplished.  Roads  are  clearly 
visible  using  lidar  and  orthophoto. 
Jeep  tracks  and  other  faint  linear 
features  were  distinguishable 
from  major  roads. 

27 

MRS  feature 
identification 

Identify  vehicle  tracks. 

Lidar  and  photo  data  sets  will 
be  examined  for  linear 
features. 

Each  lidar  and 
orthophoto  data 
set  and 
combinations. 

90%  identification 
of  field-identified 
vehicle  tracks. 

Lidar:  Unknown.  Vehicle  tracks 
were  visible  using  lidar  data 
where  such  tracks  are  deeper  than 
5-10  cm.  However,  field 
validation  was  not  conducted. 
Orthophotos:  Unknown.  Vehicle 
tracks  are  clearly  visible  including 
faint  jeep  trails.  However,  field 
validation  was  not  conducted. 

28 

MRS  feature 
identification 

Identify  topography 
that  can  limit  access. 

Lidar  data  sets  will  be  used  to 
map  areas  above  designated 
slope. 

Each  lidar  data 
set  and 
combinations. 

100%  identification 
of  steep  slope  areas. 

Accomplished.  Slopes  were 
successfully  modeled  at  both  sites 
from  lidar  data. 

Data  management 

29 

Data 

management 

Data  backup  and 
storage  to  achieve 
redundancy  and 
security. 

Data  will  be  backed  up  to 
separate  redundant  hard 
drives  or  tape  drives. 

Daily  backup 
during  field  and 
data  processing 
operations. 

Data  security, 
prevention  of  data 
loss. 

Partially  Accomplished.  All 

data  was  and  remains  backed  up 
to  redundant  hard  drives.  The 
only  exception  is  the  one 
orthophoto  image  at  Kirtland  that 
was  lost,  apparently  during 
original  data  collection  and  before 
it  was  written  to  the  hard  drive  in 
the  aircraft. 
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Table  4-2 

Data  Quality  Metrics — MRS  Identification  and  Analysis  (continued) 


1 

Analytical 

Objective 

Metric 

Action  to 

Achieve  Metric 

Sampling 
Freqnency  or 
Timing 

Desired 

Resnlt 

Actual  Result 

30 

Data  transfer 

Data  transfer  will  be  in 
appropriate  formats  and 
fde  sizes  for  ESTOP 
and  Kirtland  Air  Force 
Base  ongoing  use. 

Data  will  meet  US 

Government  Spatial  Data 
Standard  and  fully  comply 
with  Versar  electronic  data 
deliverable  specifications. 

Each  data 
transfer. 

Data  security  and 
quality. 

Accomplished.  Data  formats  met 
ESTCP  requirements.  ESTCP 
staff  requested  that  data  not  meet 
Spatial  Data  Standards,  but  only 
that  it  contain  standard  metadata 
fdes  and  use  ESTCP-provided 
data  codes.  This  was  provided. 

31 

Data 

collection 

report 

Standard  flight 
reporting  includes: 
calibration  log,  flight 
log,  QA/QC  log,  and 
site  photos 

The  data  collection  report  is  a 
standard  QA/QC  product. 

Calibration 
report:  each 
flight  day. 

Full  reporting  is  a 
required  part  of 
contract 
performance. 

Accomplished.  Calibration 
flights  were  accomplished  and 
standard  pitch,  roll  and  heading 
adjustment  values  were  calculated 
and  recorded. 

Flight  log:  each 
flight. 

Flight  log:  Full 
reporting. 

Accomplished.  Flight  log  data 
was  recorded  and  delivered. 

QA/QC  log: 
each  flight. 

QA/QC  log:  Full 
reporting. 

Accomplished.  Flight  logs  and 
QA/QC  report  were  provided. 

Site  photos: 
whole  project. 

Site  photos:  Full 
reporting. 

Accomplished.  Site  photos  were 
taken  and  delivered. 

32 

Data 

processing 

report 

Standard  data 
processing  report 
includes:  GPS  control 
ties,  accuracy 
verification  report,  and 
QA/QC  report. 

The  data  processing  report  is 
a  standard  QA/QC  product. 

Each  lidar  and 
orthophoto  data 
set. 

Full  reporting  is  a 
required  part  of 
contract 
performance. 

Accomplished.  GPS  control, 
accuracy  verification  report  and 
QA/QC  reports  were  delivered. 
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Table  4-2 

Data  Quality  Metrics — MRS  Identification  and  Analysis  (continued) 


1 

Analytical 

Objective 

Metric 

Action  to 

Achieve  Metric 

Sampling 
Freqnency  or 
Timing 

Desired 

Result 

Actual  Result 

33 

Metadata 

Metadata  to  accurately 
describe  data  format 
and  processing  steps. 

Data  will  meet  US 

Government  Spatial  Data 
Standards  and  fully  comply 
with  Versar  electronic  data 
deliverable  specifications 
including  metadata  standards. 

Each  data 
transfer. 

Metadata  meets 
required  standards. 

Accomplished.  ESTCP  staff 
stated  that  Spatial  Data  Standards 
would  not  apply  but  that  metadata 
would  be  required.  Standard  GIS 
metadata  files  were  delivered  with 
all  data. 

34 

QA/QC 

All  data  and  derived 
products  will  be  subject 
to  appropriate  QA/QC 
review. 

Data  processing  will  follow 
the  QA/QC  plan  described 
herein. 

Each  data 
transfer. 

Data  are  valid 
useful  for  the 
intended  purpose 
and  defensible. 

Accomplished.  Each  data 
deliverable  was  independently 
reviewed  by  the  GIS  Lead  and  a 
standard  QA/QC  form  filled  out 
and  placed  in  the  project  fdes. 

35 

Data  delivery 

All  data  will  be 
delivered  in  a  timely 
and  easy-to-transfer 
manner. 

Data  deliverables  will  be 
made  using  ftp  where 
possible,  but  in  all  cases  will 
be  followed  up  with  delivery 
on  physical  media,  primarily 
external  hard  drives. 

Each  data 
transfer. 

Meeting  data 

deliverable 

deadlines. 

Accomplished.  Data  was 
delivered  on  or  before  the  dates 
given  in  the  Demonstration  Plan. 
Interim  data  deliveries  were  made 
by  DVD  or  external  hard  drive. 
Final  data  delivery  was 
accomplished  through  external 
hard  drive. 
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Table  4-3 

Data  Quality  Metrics,  Individual  Performance  Measures 


1 

Analytical 

Objective 

Metric 

Action  to 

Achieve  Metric 

Sampling 
Freqnency  or 
Timing 

Desired 

Resnlt 

Actual  Resnlt 

Pre-flight  Activities 

1 

Study  area 

boundary 

delineation 

Both  Sites:  Site 
boundary  polygon 
characteristics  agreed 
on  and  documented 
to  allow  comparison 
with  data  collected. 

Achieve  stakeholder  agreement 
to  boundary  parameters. 

Once  at 
beginning  of 
program. 

Document  site 
boundary  for 
measurement  of 
future  boundary 
metrics. 

Accomplished.  Boundaries  were 
agreed  on  and  utilized. 

2 

Survey  control 
point 

confirmation 

measurement 

Both  Sites:  Confirm 
coordinates  of  survey 
control  points  within 
at  least  third-order 
accuracy. 

Perform  and  record  GPS 
survey  (static  or  kinematic). 

Pre-flight  (or 
during  on-site 
acquisition). 

Confirm 
coordinates  of 
survey  control 
points. 

Accomplished.  Control  points  were 
independently  occupied  by  TRSI  and 
ESTCP. 

Lidar  Data  Collection  and  Processing 

Sensor 

calibration 

Both  Sites:  Resolve 
roll/pitch/ 
heading  for 
installation. 

Perform  opposing  direction 
and  orthogonal  passes  over 
baseline.  Compare  with 
nominal  values  from  standard 
installation. 

Prior  to  each 
flight  day. 

+/-  0.02  degrees 

Accomplished.  Standard  roll,  pitch 
and  heading  correction  factors  were 
established  through  calibration  flights 
for  both  sites.  See  Appendix  A. 

1 

Sensor  speed 

Both  Sites:  Laser 
pulse  rate  between  50 
and  100  kHz. 

Set  laser  pulse  speed  and  the 
altitude  of  the  low  lidar  passes 
depending  on  site  conditions  to 
achieve  highest  possible  point 
density. 

Prior  to  each 
flight  day. 

Achieve  target 
sensor  speed. 

Accomplished.  Laser  pulse  rate  of  50 
kHz  was  used  at  both  sites. 
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Table  4-3 

Data  Quality  Metrics,  Individual  Performance  Measures  (continued) 


1 

Analytical 

Objective 

Metric 

Action  to 

Achieve  Metric 

Sampling 
Freqnency  or 
Timing 

Desired 

Result 

Actual  Result 

5 

Flight  altitude, 
Kirtland 

Kirtland:  Flight 
altitudes  of  900,  450, 
and  200  to  300  m. 

Establish  and  fly  appropriate 
flight  altitudes  for  the  desired 
lidar  point  densities  and 
orthophoto  pixel  sizes.  Lay  out 
a  series  of  flight  lines  for  high- 
density  lidar  collection  to  be 
able  to  respond  to  site 
conditions. 

Each  flight 
line. 

+/-  50  m  from 
planned  flight 
altitudes. 

Accomplished.  Lidar  flight  altitudes 
were  900,  450  and  300  m.  Flight 
altitudes  are  documented  through 
daily  flight  logs. 

Flight  altitude, 
Victorville 

Victorville:  Flight 
altitudes  of  450  and 

300  m. 

Establish  and  fly  appropriate 
flight  altitudes  for  the  desired 
lidar  point  densities  and 
orthophoto  pixel  sizes.  Lay  out 
a  series  of  flight  lines  for  high- 
density  lidar  collection  to  be 
able  to  respond  to  site 
conditions. 

Each  flight 
line. 

+/-  50  m  from 
planned  flight 
altitudes. 

Accomplished.  Lidar  flight  altitudes 
were  900,  450  and  300  m.  Flight 
altitudes  are  documented  through 
daily  flight  logs. 

6 

Area  coverage 

Both  Sites:  100% 
coverage  for  each 
flight. 

Establish  and  fly  flight  lines  so 
as  to  cover  the  entire  target 
area.  Data  from  each  day’s 
flights  will  be  examined  and 
data  gaps  will  be  filled. 

Each  flight. 

100%  coverage. 

Accomplished.  100%  coverage  of 
the  study  area  was  accomplished  for 
each  flight  at  both  sites. 

15%  flightline 
overlap,  50m 
over  area 

boundaries. 

Accomplished.  Flight  line  overlap 
met  specifications  for  all  flights. 
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Table  4-3 

Data  Quality  Metrics,  Individual  Performance  Measures  (continued) 


1 

Analytical 

Objective 

Metric 

Action  to 

Achieve  Metric 

Sampling 
Freqnency  or 
Timing 

Desired 

Resnlt 

Actual  Resnlt 

7 

Data  collection 
rate 

Both  Sites:  Collect 
data  for  the  entire  site 
within  the  established 
schedule.  Reserve 
one  additional  day  for 
QA/QC  and  re-flight. 

Establish  and  review  flight 
lines  and  flight  schedule  prior 
to  data  collection. 

NA 

Full  data 
collection  within 
planned 
schedule. 

Partially  Accomplished.  Data  was 
acquired  as  planned  at  Kirtland.  At 
Victorville,  data  collection  required 
two  additional  days  due  to  high 
ambient  air  temperature. 

8 

Lidar  point 
density 

Achieve  overall 
densities  of: 

Plan  and  accomplish 
appropriate  sensor  speed,  flight 
altitude,  and  air  speed.  Flights 
more  than  10%  below  target 
point  densities  will  be 
repeated. 

Each  flight. 

Data  collection 
within  10%  of 
target  densities. 

Partially  Accomplished.  At 

Kirtland,  data  for  the  900  m  and  450 
m  flights  met  or  exceeded  target 
densities.  Data  was  below  target 
density  for  the  low-level  flights. 

Lower  densities  resulted  from  50  kHz 
laser  pulse  rate  combined  with  300  m 
flight  altitude.  At  Victorville,  data 
met  target  densities. 

Kirtland  I 

200  m  flights  (2)  -  8 
pts/m^  each 

200  m  flights  (2) 

-  8  pts/m^  each 

N  block  flight  1 :  5.2  pts/m^ 

N  block  flight  2:  6.5  pts/m^ 

S  block  flight  1:  5.9  pts/m^ 

S  block  flight  2:  6.1  pts/m^ 

Kirtland  II 

450  m  flight  (1)  -  3 
pts/m^ 

450  m  flight  (1) 

-  3  pts/m^ 

N  block:  5.2  pts/m^ 

S  block:  4. 1  pts/m^ 

Kirtland  III 

900  m  flight  (1)  - 
l.SptsW. 

900  m  flight  (1) 

-  1.5pts/m^. 

N  block:  1 .4  pts/m^ 

S  block:  1.6  pts/m^ 
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Table  4-3 

Data  Quality  Metrics,  Individual  Performance  Measures  (continued) 


1 

Analytical 

Objective 

Metric 

Action  to 

Achieve  Metric 

Sampling 
Freqnency  or 
Timing 

Desired 

Resnlt 

Actual  Resnlt 

Victorville  I 

300  m  flight  -  6 
pts/m^ 

300  m  flight  -  6 
pts/m^ 

6.4  pts/m^ 

Victorville  II 

450  m  flight  -  4.5 
pts/m^ 

450  m  flight  - 
4.5  ptsW 

4.8  ptsW 

9 

Lidar  flight 
line  alignment 

Kirtland  The  two 

200  m  flights  will  be 
orthogonal. 

Appropriate  flight  lines  will  be 
designed  and  flown.  Planned 
flight  lines  will  be  submitted  in 
advance. 

Each  flight. 

Flight  lines 
within  10°  of 
orthogonal. 

Partially  Accomplished.  North 
block  flights  were  orthogonal.  South 
block  flights  were  both  flown 
northeast-southwest.  The  Double 

Eagle  airport  runway  is  just  north  of 
the  south  block.  North-south  flight 
lines  would  have  required  crossing 
and  turning  directly  over  the  airstrip, 
which  was  impossible  for  safety 
reasons. 

Victorville:  The  300 
m  and  450  m  flights 
will  be  orthogonal. 

Appropriate  flight  lines  will  be 
designed  and  flown.  Planned 
flight  lines  will  be  submitted  in 
advance. 

Each  flight. 

Flight  lines 
within  10°  of 
orthogonal. 

Accomplished.  The  two  flights  were 
flown  orthogonal. 
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Table  4-3 

Data  Quality  Metrics,  Individual  Performance  Measures  (continued) 


1 

Analytical 

Objective 

Metric 

Action  to 

Achieve  Metric 

Sampling 
Freqnency  or 
Timing 

Desired 

Resnlt 

Actual  Result 

10 

Lidar  vertical 
accuracy 

Both  Sites:  Vertical 
accuracy  of+/-  15 
cm  compared  to 
ground  survey. 

Steps: 

Each  flight 

1 .  Perform  sensor  calibration 
as  described  above 

As  above. 

Accomplished. 

2.  Obtain  ground  elevations  on 
identifiable  points  using 
ground  based  GPS  methods 
(static  and/or  kinematic) 

As  above. 

Accomplished.  Conbol  points  were 
collected  as  described. 

3.  Compare  ground-based  and 
airborne  elevations. 

Meet  or  exceed 
+/1  15  cm  lidar 
to  control  point 
vertical 
accuracy. 

Accomplished.  Elevation 
comparisons  were  performed  between 
conbol  point  elevations  and  the 
interpolated  elevation  of  the  lidar 
surface  at  that  point.  Results  were 
within  specifications.  See 

Appendix  A. 

11 

Lidar 

horizontal 

accuracy 

Both  Sites: 

Horizontal  accuracy 
of  +/-  65  cm 
compared  to  ground 
survey. 

Steps: 

Each  flight. 

Accomplished.  Positions  of  control 
points  were  obtained  in  the  Lidar  data 
using  intensity  values.  These 
positions  were  compared  to  the 
surveyed  locations  of  these  conbol 
points.  Horizontal  accuracy  was  well 
within  specification.  See 

Appendix  A. 

1 .  Perform  sensor  calibration 
as  described  above 

As  above. 

Accomplished. 
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Table  4-3 

Data  Quality  Metrics,  Individual  Performance  Measures  (continued) 


1 

Analytical 

Objective 

Metric 

Action  to 

Achieve  Metric 

Sampling 
Frequency  or 
Timing 

Desired 

Result 

Actual  Result 

2.  Obtain  ground  positions  on 
identifiable  points  using 
ground  based  GPS  methods 
(static  and/or  kinematic) 

As  above. 

Accomplished. 

3.  Compare  ground-based  and 
airborne  positions. 

Meet  or  exceed 
+/-  65  cm  lidar  to 
control  point 
horizontal 
accuracy. 

Accomplished. 

12 

Lidar  data 
integration  - 
flight  lines 

Both  Sites:  Achieve 
flight  line  to  flight 
line  edge  match  of 
+!-  12cm. 

Review  statistics  from  lidar 
processing  software. 

Line  to  line. 

Achieve  best 
possible  match 
between 
individual  lidar 
flight  lines. 

Accomplished. 

13 

Lidar  point 
separation 

Both  Sites:  Remove 
100%  of  large 
features,  (trees, 
buildings,  vehicles) 

Operators  remove  non-ground 
laser  returns  through 
automated  separation  routines 
followed  by  hand  cleaning  and 
inspection. 

Lidar  data  set 
for  each  flight. 

Satisfactory 
visual  inspection 
of  surface  model 
of  the  ground 
surface. 

Accomplished.  Lidar  points  were 
classified  as  ground  and  non-ground 
returns.  Visual  inspection  of  the 
ground  returns  showed  that  all 
buildings,  fences,  and  other  larger 
features  were  successfully  removed. 

Remove  small 
features  (grass,  low 
brush)  to  the  level 
where  remaining  data 
cannot  distinguish 
ground  from  non¬ 
ground  features. 

Satisfactory 
visual  inspection 
of  surface  model 
of  the  ground 
surface. 

Accomplished.  Inspection  of  ground 
and  non-ground  lidar  points  in 
conjunction  with  orthophotos  showed 
that  small  brush  and  tall  grass  was 
removed  within  specification. 
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Table  4-3 

Data  Quality  Metrics,  Individual  Performance  Measures  (continued) 


1 

Analytical 

Objective 

Metric 

Action  to 

Achieve  Metric 

Sampling 
Freqnency  or 
Timing 

Desired 

Resnlt 

Actual  Result 

Orthophoto  Data  Collection  and  Processing 

14 

Orthophoto 
area  coverage 

Both  Sites:  100% 
coverage  for  each 
flight. 

Wireframes  of  “raw”  images 
are  compared  to  the  project 
boundary  to  check  for  gaps  or 
holes. 

Each  flight  day 
as  part  of 
QA/QC 
checks. 

100%  coverage 
with  sufficient 
image  overlap 
for  ortho¬ 
rectification. 

Partially  Accomplished.  At 

Kirtland,  complete  coverage  was 
obtained  for  both  flights  with  the 
exception  of  one  image  in  the  10  cm 
flight.  The  missing  image  is 
approximately  125  x  200  meters, 
located  on  the  north  boundary  of  the 
study  area.  The  file  appears  to  have 
been  corrupted  during  collection.  At 
Victorville,  complete  coverage  was 
obtained. 

15 

Orthophoto 
flight  altitude/ 
pixel  size 

Kirtland  I 

450  m  (for  10  cm 
pixel  flight) 

Orthophoto  pixel  size  is 
directly  related  to  flight 
altitude.  Flight  lines  are 
designed  for  the  desired  pixel 
sizes.  Flight  data  will  be 
examined  during  and  after  each 
flight  and  flight  lines  outside  of 
the  range  will  be  repeated. 

Each  flight. 

Achieve 
specified  flight 
altitudes  and 
pixel  sizes. 

Accomplished.  10  cm  pixel  sizes 
were  achieved  for  100%  of  the  study 
area  with  the  exception  of  the  one 
missing  image. 
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Table  4-3 

Data  Quality  Metrics,  Individual  Performance  Measures  (continued) 


1 

Analytical 

Objective 

Metric 

Action  to 

Achieve  Metric 

Sampling 
Freqnency  or 
Timing 

Desired 

Result 

Actual  Result 

Kirtland  II 

900  m  (for  20  cm 
pixel  flight). 

Achieve 
specified  flight 
altitudes  and 
pixel  sizes. 

Accomplished.  20  cm  pixel  sizes 
were  achieved  for  100%  of  the  study 
area. 

Victorville 

450  m  (for  10  cm 
pixel  flight) 

Achieve 
specified  flight 
altitudes  and 
pixel  sizes. 

Accomplished.  10  cm  pixel  sizes 
were  achieved  for  100%  of  the  study 
area. 

16 

Orthophoto 

image 

mosaicing 

Both  Sites:  No 

obvious  seams 
between  images  in 
the  final  orthophoto. 

Creation  of  an  image  mosaic 
from  individual  small  images 
is  largely  an  operator 
controlled  rather  than  an 
automated  process. 

Each 

orthophoto 
composite 
image  (10  cm 
and  20  cm). 

Line  features  are 
continuous  with 
no  visible 
discontinuity  at 
mosaic  seams. 

Accomplished.  Visual  inspection  of 
the  orthophoto  images  showed  no 
obvious  seams. 

17 

Orthophoto 
image  color 
balancing 

Both  Sites:  No 

obvious  color 
imbalances  within 
data  for  each  session. 

Color  balancing  is  an  operator 
controlled  process  based  on 
viewing  the  mosaic  to  identify 
any  areas  of  tonal  imbalance. 

Each 

orthophoto 
composite 
image  (10  cm 
and  20  cm). 

Continuity  of 
tone  such  that 
individual 
images  are  not 
visible  in 
mosaic. 

Accomplished.  Visual  inspection  of 
the  orthophoto  images  showed  no 
obvious  color  imbalances. 

18 

Orthophoto 
horizontal 
alignment  to 
lidar 

Both  Sites:  Lidar  and 
orthophotos  aligned 
so  that  target  features 
are  not  displaced  in 
the  two  data  sets. 

Orthorectification  is  performed 
using  the  lidar  data  and  fiducial 
locations  are  control  data 
sources,  followed  by  operator 
adjustment. 

Each 

orthophoto 

composite 

image. 

Orthophotos 
aligned  to  +/-  2 
pixel  widths. 

Accomplished.  Positions  of  control 
targets  were  compared  using  the 
orthophoto  and  lidar  intensity  values. 
Locations  were  within  specifications. 
See  Appendix  A. 
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Table  4-3 

Data  Quality  Metrics,  Individual  Performance  Measures  (continued) 


1 

Analytical 

Objective 

Metric 

Action  to 

Achieve  Metric 

Sampling 
Freqnency  or 
Timing 

Desired 

Resnlt 

Actual  Result 

19 

Orthophoto 
horizontal 
alignment  to 
fiducials 

Both  Sites: 

Orthophotos  aligned 
to  survey  control 
points  so  that  target 
features  are  not 
displaced. 

Orthorectification  is  performed 
using  the  lidar  data  and  fiducial 
locations  are  control  data 
sources,  followed  by  operator 
adjustment. 

Each 

orthophoto 

composite 

image. 

Orthophotos 
aligned  to  +/-  3 
pixel  widths. 

Accomplished.  Ortho  image 
positions  were  compared  to  control 
targets  visible  in  the  images.  In 
addition,  lidar  and  orthophoto 
positions  were  compared  for  building 
comers  and  edges  of  pavement  that 
were  visible  in  both  the  orthophoto 
and  lidar  data.  Positions  were  within 
specifications.  See  Appendix  A. 
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4.2  PERFORMANCE  CONFIRMATION  METHODS 

4.2.1  Demonstration-Level  Confirmation  Methods 

At  the  demonstration  level,  the  effeetiveness  of  lidar  and  orthophotos  was  evaluated  based  on  its 
ability  to  meet  the  stated  performanee  eriteria  given  in  Table  4-1 .  The  demonstration  relied  on 
proven  industry  methods  to  assure  predietable  results,  ineluding  the  use  of  survey  eontrols, 
equipment  ealibration,  alignment  of  lidar  points  to  the  survey  eontrol  points  and  from  one  flight 
line  to  the  next,  and  QA/QC  eheeks  throughout  the  projeet.  Both  lidar  and  orthophoto  data  met 
all  data  quality  speeifieations. 

During  data  analysis,  standard  GIS  methods  were  used  to  ereate  surfaee  models  from  the  lidar 
data.  As  a  means  to  ensure  objeetive  results,  staff  assigned  to  initial  feature  identifieation  was 
not  provided  with  the  CSM  and  did  not  know  the  history  of  munitions  use  at  the  site.  All  feature 
and  MRS  identifieation  was  subjeet  to  QA/QC  review  by  projeet  staff  familiar  with  both  the 
CSM  and  UXO/MEC  generally. 

4.2.2  Program  Level  Confirmation  Methods 

Lidar  and  orthophotos  were  evaluated  for  their  ability  to  meet  the  goals  of  the  WAA  Pilot 
Program  as  a  whole.  Evaluation  was  based  on  eomparing  the  results  of  the  lidar  and  orthophoto 
data  with  the  results  of  the  other  data  eolleeted  at  the  two  sites,  ineluding  helieopter  and  transeet 
magnetometry  and  transeet-based  EMI,  along  with  validation  data  ineluding  100%  eoverage 
magnetometry  areas,  site  reeonnaissanee,  and  intrusive  sampling.  In  general,  these  subsequent 
aetivities  eonfirmed  the  findings  and  usefulness  of  lidar  and  orthophotos.  They  also  revealed 
areas  where  lidar  and  orthophotos  did  not  deteet  areas  of  eoneentrated  munitions  use,  with 
implieations  for  the  appropriate  use  of  these  teehnologies  at  future  sites. 

Eor  both  of  the  sites,  lidar,  orthophotos,  helieopter  magnetometry,  and  towed-array 
magnetometry  or  EMI  were  eolleeted  on  the  entire  demonstration  site  where  terrain  and 
vegetation  allowed.  This  proeedure  was  designed  to  faeilitate  eomparison  of  the  eontributions  of 
the  various  teehnologies  in  different  site  eonditions.  At  produetion  sites,  it  is  assumed  that  the 
three  teehnologies  would  be  deployed  in  a  manner  that  would  eonsider  time  and  budget 
eonstraints  faeed  by  site  managers.  In  sueh  a  model,  lidar  and  orthophotos  would  be  deployed  to 
the  site  first  and  the  results  used  to  guide  the  deployment  of  the  subsequent  magnetometry  layers. 

Validation  aetivities  ineluded  100%  eoverage  surveys  of  small  pareels  using  the  ground  system, 
reeonnaissanee  surveys,  and  a  limited  intrusive  investigation.  These  were  undertaken  to  validate 
both  the  data  eolleeted  by  the  other  teehnologies,  and  their  eonelusions,  sueh  as  the  eonelusions 
as  to  the  ordnanee  origin  of  features. 
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The  reconnaissance  survey  team  (an  explosive  ordnance  disposal  specialist  and  a  geophysicist 
with  a  hand-held  GPS  and  camera)  located  and  confirmed  anomalies  observed  in  the  helicopter 
data.  The  ground  systems  were  used  to  survey  smaller  (tens  of  acres)  patches  near  targets  and  in 
areas  of  no  obvious  munitions  use,  to  determine  the  distributions  of  anomalies  across  the  site. 
The  intrusive  investigation  confirmed  the  identity  of  sub-surface  features  detected  by  the 
geophysical  sensors  and  confirmed  the  reliability  of  feature  classification  using  the  geophysical 
results. 

4,2,3  Results:  Kirtland 

At  the  Kirtland  site,  helicopter  and  ground-based  magnetometry  confirmed  the  ordnance  origin 
of  the  MRS  identified  in  the  initial  CSM.  There  was  good  general  agreement  between  the  lidar 
and  orthophoto  data  and  the  helicopter  magnetrometry  data,  as  shown  in  Figure  4-1. 


Figure  4-1 

Kirtland  Helicopter  Magnetometry  Density  Grid 


Source:  ESTCP  (2007a) 
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Results  for  the  individual  targets  and  areas  of  interest  (AOIs)  are  diseussed  below. 

Target  N3 

Target  N3  is  the  most  eomplex  target  area  at  the  Kirtland  demonstration  site.  For  the  main 
bombing  target,  the  lidar  and  orthophoto  data  were  very  eonsistent  with  the  validation  data. 

Lidar  and  orthophoto  data  elearly  showed  the  eoneentrie  cireles  of  the  bombing  target.  On-site 
reeonnaissanee  loeated  the  target  eenter  but  not  the  rings,  whieh  were  too  eroded  to  be  visible 
from  the  ground.  Helieopter  magnetometry  showed  coneentrations  of  anomalies  within  the 
target  rings,  as  did  the  transeet  data.  Intrusive  sampling  loeated  abundant  ordnanee-related  serap 
along  with  four  intaet  ordnanee  items. 

There  were  no  apparent  eraters  within  the  aiming  eireles,  and  all  ordnanee-related  serap  retrieved 
from  the  intrusive  sampling  was  from  100-pound  M38A2  praetiee  rounds.  The  arehival  seareh 
report  did  not  indieate  any  other  munitions  activity  at  this  target.  All  data  is  consistent  the  CSM, 
which  identified  the  area  as  a  practice  bombing  target. 

There  were  eight  additional  AOIs  at  Target  N3,  six  of  which  were  identified  from  the  lidar  and 
orthophoto  data  and  two  that  were  identified  from  the  magnetometry  data.  The  results  using 
lidar  and  orthophotos,  as  well  as  from  the  relevant  subsequent  investigations  are  given  below, 
using  the  AOI  names  given  in  the  revised  CSM; 

•  N3-AOI-1  is  the  possible  crossed  runway  target  southwest  of  the  outer  aiming  circle. 
This  area  was  identified  in  the  lidar  data  as  a  potential  additional  target  area.  Portions 
of  this  area  showed  concentrations  of  geophysical  anomalies. 

•  N3-AOI-2  and  N3-AOI-3  were  identified  from  the  lidar  data  by  possible  craters.  Site 
reconnaissance  identified  these  as  depressions  with  no  evidence  of  ordnance  scrap. 

•  N3-AOI-4  appears  in  the  lidar  data  as  a  diamond-shaped  potential  target  area  west  of 
the  main  target.  (The  CSM  describes  this  as  a  “raised  area  shaped  in  a  cloverleaf 
pattern.”)  There  were  high  concentrations  of  geophysical  anomalies  throughout  the 
area  and  a  large  amount  of  ordnanee-related  scrap  was  retrieved. 

•  N3-AOI-5  appears  in  the  lidar  data  as  a  long  rectangular  disturbance  of  the  ground 
shaped  like  a  single  runway.  Few  anomalies  were  seen  in  the  helicopter  or  transect 
magnetometry  data;  however,  anomalies  appeared  in  the  100%  geophysical  survey 
and  some  ordnanee-related  scrap  was  retrieved  during  the  intrusive  sampling. 

•  N3-AOI-6  appears  in  the  lidar  data  ship-shaped  potential  target.  The  shape  was 
confirmed  by  site  reconnaissance,  while  geophysical  data  showed  high  concentrations 
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of  anomalies  and  intrusive  sampling  located  ordnance-related  scrap  and  one  intact 
ordnance  item. 

•  N3-AOI-7  and  N3-AOI-8  were  not  identified  using  the  lidar  or  orthophoto  data  and 
were  identified  using  helicopter  magnetometry  by  concentrations  of  geophysical 
anomalies.  Intrusive  investigation  showed  that  both  areas  had  large  amounts  of 
ordnance-related  scrap,  and  three  intact  ordnance  items  were  located  within 
N3-AOI-7. 


The  results  of  these  activities  at  the  AOIs  are  shown  in  Figures  4-2  and  4-3. 


Figure  4-2 

Kirtland  Target  N3  and  Associated  AOIs 


Areas  ot  Interest 


Identified  in  CSM  ver 0 

ESTCP  Identified  Features 

URS  Identified  Features 

Kirtland  Precision  Bombing  Range 

N-3  Target  Area  -  2005  Orthophoto  and  LiDAR 

0  95  190  380  Meters 

1  1  1  1  1  1  1  1  1 

Source:  ESTCP  (2007a) 

The  location  of  N3-AOI-7  and  N3-AOI-8,  which  were  not  detected  using  lidar  and  orthophotos, 
are  shown,  just  southeast  and  southwest  of  the  aiming  circles. 
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Figure  4-3 

Kirtland  Target  N3  and  Associated  AOIs,  Geophysics  Data 


Souree:  ESTOP  (2007a) 
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Simulated  Oil  Refinery 

This  SORT  target  is  described  in  the  initial  CSM  as  a  series  of  cells  forming  a  rectangular  shape. 
The  target  outline  shown  in  Figures  4-4  and  4-5  is  based  on  interpretation  of  the  lidar  and 
orthophoto  data,  in  which  rectangular  cells  and  other  berms  are  visible.  The  helicopter  and 
transect  magnetometry  data  showed  a  concentration  of  anomalies  only  in  the  western  portion  of 
the  target  area.  The  lack  of  craters  and  the  nature  of  the  ordnance  scrap  recovered  support  the 
conclusion  that  only  100-pound  practice  bombs  were  used  at  this  target. 


Figure  4-4 

Kirtland  Target  SORT,  Geophysics  Data 


□□  Area  of  interest  •  Anomaly 
□  Target.Airjng  Circle  TransectLine 


Kirtland  PBR 
Oil  Refinery  Target  Area 
Geophysical  Methods  Data 


0  105  210  420  Meters 

I _ I _ I _ I _ I _ I _ I _ I _ I 


Source:  ESTCP  (2007a) 


The  transect  and  100%  coverage  magnetometry  data  showed  that  anomalies  extended  past  the 
boundaries  of  the  target  visible  in  the  lidar  and  orthophoto  data,  though  at  low  concentrations. 
Intrusive  surveys  confirmed  that  these  anomalies  included  ordnance-related  scrap,  which 
extended  to  the  farthest  area  surveyed. 
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Figure  4-5 

Kirtland  Target  SORT,  Intrusive  Investigation  Results 
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Kirtland  PBR  Area  2  (abc)  Total  Coverage 
Intrusive  Investigation  Results 

Souree:  ESTOP  (2007a) 
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Target  N2 

Target  N2  is  described  as  a  1,000-foot  bulTs-eye  target  used  for  100-pound  practice  bombs. 

Lidar  and  orthophoto  data  clearly  showed  the  concentric  circles  making  up  this  bombing  target, 
although  these  were  not  visible  from  ground  reconnaissance.  No  craters  were  seen  in  either  the 
lidar  data  or  site  reconnaissance.  Helicopter  and  ground-based  magnetometry  showed  high 
concentrations  of  anomalies  throughout  the  target  circle  to  the  outer  ring  (Figure  4-6).  A  large 
amount  of  small  frag  and  ordnance-related  scrap  was  observed  at  the  surface.  Ordnance-related 
scrap  retrieved  as  part  of  the  intrusive  investigation  was  all  from  100-pound  M38A2  practice 
rounds  or  aerial  flares.  This  data  is  all  consistent  with  the  conclusion  that  the  target  was  used  for 
practice  bombing. 

A  structure  located  on  the  second  inner  ring  to  the  north  of  the  center  is  visible  in  both  the 
orthophotography  and  the  lidar  data.  Site  reconnaissance  identified  this  a  razed  cinderblock 
building. 

The  results  of  100%  magnetometry  surveys  and  intrusive  investigation  were  consistent  with  each 
other.  There  was  a  high  concentration  of  ordnance-related  anomalies  and  objects  within  the 
target,  with  the  density  falling  off  with  distance  but  not  to  zero.  These  results  indicate  that,  as  at 
Target  SORT,  even  the  area  considered  background  contained  some  ordnance-related  material. 

Figure  4-6 

Kirtland  Target  N2,  Geophysics  Data 


1  1  ftrea  of  Interest  *  Anomaly 

Target  Ai  rnng  Circle  TransectLne 

Kirtland  PBR 

N-2  Target  Area 
Geophysical  Methods  Data 

0  25  50  100  Meters 

I  I  I  I  I  :  I _ I 

Source:  ESTCP  (2007a) 
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New  Demolitions  Impact  Area 

Target  NDIA  is  described  in  the  initial  CSM  as  a  1,000-foot-diameter  HE  bull’s-eye  target.  The 
lidar  data  did  not  show  any  evidence  of  target  rings,  but  did  show  numerous  potential  craters. 

The  orthophoto  data  showed  target  cross-hairs  and  one  target  ring.  Helicopter  and  transect 
magnetometry  both  showed  a  low  density  of  anomalies  in  and  near  the  target  area  (Figure  4-7). 
The  100%  ground  geophysical  surveys  collected  through  the  center  of  the  inner  target  circle  and 
at  the  northern  edge  of  the  outer  circle  showed  concentrated  anomalies  at  the  center,  getting  more 
sparse  further  away.  Intrusive  sampling  was  not  performed  on  these  anomalies  due  to  the 
possible  presence  of  250-pound  HE  bombs  in  close  proximity  to  the  airport.  Field 
reconnaissance  located  craters  and  heavy  walled  frag.  Both  indicate  of  the  use  of  HE  at  this  site. 

Figure  4-7 

Kirtland  Target  NDIA,  Geophysics  Data 
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Additional  Areas 

Eleven  additional  AOIs  were  identified  at  the  Kirtland  site,  nine  of  them  in  the  southern  portion 
of  the  study  area  and  two  in  the  northern  portion  near  Target  N2.  The  archival  search  report  did 
not  indicate  any  munitions-related  activities  in  these  areas,  and  subsequent  surveys  found  all  but 
one  of  these  AOIs  to  be  building  foundations,  animal  burrows,  or  other  areas  with  no  evidence  of 
ordnance  use.  The  one  exception  was  the  potential  bull’s-eye  target  described  in  Section  3.7.9, 
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which  was  found  to  contain  a  small  number  of  anomalies:  two  rings  of  light-eolored  pebbles  and 
a  small  amount  of  ordnanee-related  serap.  The  revised  CSM  deseribes  this  as  a  possible  aiming 
circle,  with  a  medium  confidence  level.  The  site  is  loeated  relatively  close  to  a  known  target 
area  just  outside  of  the  study  area,  and  it  is  possible  that  this  is  the  souree  of  the  ordnance-related 
serap  found. 

4,2,4  Results:  Victorville 
Target  DBT  Y 

Target  DBT  Y  is  deseribed  as  a  HE  bombing  target,  and  large  eraters  were  elearly  shown  using 
both  lidar  and  orthophotos.  Helicopter  and  ground-based  magnetometry  and  EMI  found  few 
anomalies  at  the  site,  and  there  was  little  evidence  of  munitions-related  scrap.  The  revised  CSM 
states  that  this  is  nevertheless  consistent  with  potential  HE  use,  which  can  leave  few  items  large 
enough  to  result  in  either  anomalies  or  visible  scrap.  The  site  has  been  the  site  of  previous 
cleanup  aetivities  whieh  may  also  account  for  the  seareity  of  items.  Heavy-walled  serap 
indieative  of  HE  use  was  found  just  outside  of  the  dry  lake  bed  near  similar  craters  to  those  at  the 
site. 

Target  PBR  15 

Target  PBR  15  is  described  as  a  bull’s-eye  target  used  for  precision  bombing  practice. 
Orthophotos  and  lidar  intensity  imagery  elearly  showed  the  bull’s  eye  rings.  Coneentrations  of 
anomalies  from  both  helicopter  and  ground-based  magnetometry  were  found  up  to  a  circle  of 
1 ,000  feet  in  diameter  surrounding  the  target.  The  lack  of  cratering  at  the  target  supported  the 
eonelusion  that  the  target  was  used  for  practiee  bombing. 

Eigure  4-8  shows  the  ground  magnetometry  transect  data  as  filtered  for  geologic  noise.  Target 
PBR  15  is  elearly  visible  in  the  southeast  portion  of  the  site.  Target  DBT  Y  is  faintly  visible. 
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Figure  4-8 

Victorville  Filtered  Geophysics  Data 
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Additional  AOIs 

Figure  4-9  shows  the  additional  AOIs  identified  at  the  Vietorville  site.  All  but  the  eraters  west  of 
the  dry  lake  bed  were  identified  based  on  the  magnetometry  and  EMI  surveys.  Lidar  data 
showed  a  number  of  apparent  craters  west  of  the  dry  lake  bed,  and  ESTOP  conducted  site 
reconnaissance  of  some  of  these  features  (labeled  in  the  revised  CSM  as  DBTY-AOI-1).  The 
presence  of  heavy- walled  ordnance  frag  confirmed  that  some  of  these  features  were  craters  from 
HEs.  However,  the  small  number  of  craters  found  suggested  that  these  craters  were  from  pilot 
error  rather  than  a  separate  target.  Others  features  to  the  far  west  in  this  area  were  identified  as 
mining  prospect  pits  or  large  boulder  formations.  The  other  AOIs  were  examined  with  ground- 
based  magnetometry  and  EMI  sensing,  and  were  determined  not  to  be  ordnance-related. 
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Figure  4-9 

Victorville  Helicopter  Magnetometry  Density  Grid 
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4,2,5  Findings  and  Conclusions 

Examination  of  the  lidar  and  orthophoto  data  in  light  of  the  subsequent  aetivities  at  the  two  sites 
led  to  the  following  findings  and  eonelusions: 

•  Lidar  and  orthophotos  located  all  of  the  major  target  features.  The  general  locations  of 
these  targets  were  consistent  between  the  lidar  and  orthophoto  data  and  that  of  the  other 
technologies  used. 

•  Lidar  and  orthophotos  did  not  locate  two  ordnance-related  areas  associated  with  Kirtland 
Target  N3,  presumably  because  these  areas  did  not  show  ground  disturbances.  These 
areas  were  detected  using  all  of  the  magnetometry  platforms,  and  were  verified  through 
intrusive  investigation.  This  result  points  out  a  limitation  of  these  light-based 
technologies:  because  they  do  not  direct  ordnance  components  directly,  they  can  miss 
areas  of  munitions  use  that  do  not  leave  evidence  on  the  ground  surface.  Consequently, 
use  of  lidar  and  orthophotos  should  be  supplemented  with  other  technologies  where 
appropriate. 
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•  Lidar  and  orthophotos  tended  to  understate  the  extent  of  distribution  of  ordnanee-related 
scrap  at  the  Kirtland  site.  Ordnance-related  scrap  continued  to  be  found  in  most  of  the 
areas  of  100%  coverage  and  intrusive  investigation,  well  past  what  would  have  been 
estimated  as  conservative  boundaries  of  the  features  using  the  lidar  and  orthophoto  data. 
The  reason  for  this  wide  distribution  of  ordnance  scrap  is  not  known.  Because  no 
intrusive  sampling  was  done  at  the  Victorville  site,  it  is  unknown  whether  this  pattern 
would  be  found  at  the  targets  there.  It  is  possible  that  methods  could  be  developed  to 
better  extrapolate  from  features  shown  in  lidar  to  the  actual  extent  of  ordnance-related 
scrap.  However,  this  would  require  the  examination  of  both  lidar  and  orthophoto  as  well 
as  validation  data  from  a  wider  variety  of  sites. 

•  Lidar  and  orthophotos  found  targets  using  berms  or  craters  that  would  not  have  been 
inferred  from  helicopter  magnetometry  data  alone,  including  the  main  target  ring  at 
Kirtland  Target  N3  and  Victorville  Target  DBT  Y. 

4,3  DATA  ANALYSIS,  INTERPRETATION,  AND  EVALUATION 

4,3,1  Correlations  with  Operating  Parameters  and  Required  Performance  Specifications 

Of  the  operating  parameters  tested,  lidar  and  orthophoto  data  density  showed  the  strongest 
correlation  with  operational  results.  Of  the  two  orthophoto  pixel  sizes  tested,  the  10  cm  pixel 
images  clearly  outperformed  the  20  cm  images,  allowing  the  detection  of  objects  at  or  below  1  m 
in  size.  This  is  a  significant  result  for  two  reasons:  first,  because  the  smaller  pixel  size  is  more 
expensive  to  acquire,  and  second,  because  both  10  cm  and  20  cm  pixel  orthophotos  are  more 
dense  than  conventionally  available  orthophotography,  which  very  seldom  has  pixel  sizes 
smaller  than  32  cm  (1  foot). 

Lidar  data  density  clearly  affected  the  detection  of  smaller  craters.  This  effect  was  not  as  strong 
for  the  bombing  targets,  which  were  well  detected  using  all  but  the  lowest-density  lidar  data 
collected  of  1.5  pts/  m^.  At  this  level,  bull’s-eye  target  rings  began  to  be  missed. 

At  future  sites  where  detection  of  large  features  such  as  bombing  targets  is  the  principle 
objective,  lidar  data  densities  would  not  need  to  be  as  high  as  for  reliable  detection  of  craters. 
Nevertheless,  additional  lidar  density  should  probably  be  acquired  if  the  project  budget  and  the 
vendor’s  equipment  allow,  since  additional  lidar  density  will  help  to  define  features  more  clearly. 
Additionally,  this  demonstration  showed  that  lidar  data  density  varied  considerably  over  the 
project  area.  Increasing  overall  lidar  density  is  one  way  to  compensate  for  the  possibility  that 
some  areas  will  be  less  dense  than  planned. 


Data  from  this  demonstration  also  suggests  that  if  possible,  increased  lidar  density  should  be 
acquired  through  use  of  faster  sensors  or  lower  flight  elevations,  rather  than  through  multiple 
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overlapping  flights.  This  is  because  in  areas  of  flightline  overlap,  current  equipment 
characteristics  and  data  processing  methods  exclude  significant  numbers  of  lidar  points  from  the 
ground  surface  model.  In  areas  with  very  sparse  vegetation,  the  point  classification  problem  can 
be  eliminating  by  making  surface  models  using  all  lidar  points,  however  this  approach  cannot  be 
used  on  vegetated  sites.  Further  investigation  is  appropriate  in  this  area. 

Flight  line  orientation  also  had  a  detectable  effect  on  detection  of  very  shallow  linear  features, 
indicating  that  if  a  second  lidar  flight  is  collected  in  addition  to  the  primary  lidar/orthophoto 
flight,  it  should  be  flown  perpendicular  to  the  first  flight. 

4,3,2  Optimum  Operating  Conditions  and  Appropriate  Uses  of  the  Technologies 

Results  from  the  Kirtland  and  Victorville  sites  support  the  general  premise  of  the  WAA  Pilot 
Program:  lidar  and  orthophotos  should  be  the  first  technologies  to  be  deployed  after  completion 
of  the  archival  search  report  and  the  initial  CSM.  At  both  sites,  lidar  and  orthophotos  were 
successful  at  revealing  and  verifying  the  broad  picture  of  munitions  use.  Lidar,  especially,  was 
very  successful  at  delineating  targets  and  crater  fields,  as  well  as  additional  areas  of  interest  that 
warranted  investigation.  Areas  of  interest  that  warranted  further  investigation  were  delineated. 
The  two  technologies  complemented  each  other  well,  each  providing  data  that  the  other  did  not. 
Since  vendors  generally  offer  the  two  technologies  together,  it  makes  sense  to  acquire  both  at 
future  production  sites. 

At  the  Kirtland  site,  two  AOIs  were  identified  using  magnetometry  that  were  not  detected  using 
lidar  and  orthophotos,  presumably  because  these  areas  did  not  leave  any  indications  on  the 
ground  surface.  This  finding  supports  the  WAA  Pilot  Program  approach  that  lidar  and 
orthophoto  acquisition  should  be  followed  with  technologies  that  directly  detect  ordnance 
components. 
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5.0  COST  ASSESSMENT 


5.1  COST  REPORTING 

Table  5-1  presents  aetual  eosts  for  the  Kirtland  and  Vietorville  demonstration  sites,  and 
estimated  eosts  for  produetion  sites  of  three  different  sizes.  Cost  figures  for  the  18,000-aere 
demonstration  site  are  based  on  preliminary  projeet  eosts  for  the  Former  Camp  Beale  site, 
aequired  in  2006.  The  remaining  figures  are  planning-level  estimates,  assessed  to  be  aeeurate 
within  +/-  20%.  Per-aere  eosts  for  the  Kirtland  site  were  higher  sinee  four  rather  than  two  lidar 
flights  were  eondueted,  and  one  rather  than  two  orthophoto  sets  were  ereated.  The  Vietorville 
eonfiguration,  with  one  lidar/orthophoto  flight  and  one  additional  lidar-only  flight,  is  eonsidered 
representative  for  a  produetion  site  where  it  is  important  to  deteet  both  targets  and  individual 
small  features.  The  “50,000-aere  produetion  site”  estimate  is  based  on  URS’  previous 
experienee  and  interviews  with  industry  sourees,  and  the  “1 15,000-aere  produetion  site”  estimate 
is  based  on  a  eost  proposal  made  by  URS  for  a  site  in  the  western  US  in  fall  2005.  All  projeets 
listed  ean  be  eompleted  in  less  than  one  year;  therefore,  no  diseount  faetor  has  been  applied  to 
the  figures. 


Table  5-1 

Actual  and  Projected  Costs 


Project  Parameters 

Kirtland 

Victorville 

18,000-acre 
Production  Site 

50,000-acre 
Production  Site 

115,000-acre 
Production  Site 

Project  area  size  (acres) 

5,000 

5,640 

18,000 

50,000 

115,000 

Project  area  size  (hectares) 

1,914 

2,282 

7,284 

19,140 

44,022 

Lidar  flights: 

300  m  (Lidar  only) 

2 

1 

1 

1 

1 

450  m  (Lidar  and  10  cm  pixel  imagery) 

1 

1 

1 

1 

1 

900  m  (Lidar  and  20  cm  pixel  imagery) 

1 

0 

0 

0 

0 

Total  Lidar  flights 

4 

2 

2 

2 

2 

Total  Lidar  point  density  (pts/m^) 

20 

8-10 

8-10 

8-10 

8-10 

Orthophoto  pixel  size  (cm) 

1 0  and  20 

10 

10 

10 

10 

1  Costs  (2006  $) 

Fixed  Costs 

Mob/demob 

15,600 

23,100 

21,800 

30,000 

45,000 

Planning/preparation 

15,000 

9,200 

15,000 

15,000 

20,000 

Project  management 

15,000 

10,000 

25,000 

40,000 

100,000 

Site  work 

0 

0 

0 

0 

0 

Equipment  cost 

0 

0 

0 

0 

0 

Start-up  and  testing 

0 

0 

0 

0 

0 

Subtotal  fixed  costs 

45,600 

42,300 

61,800 

85,000 

165,000 

Variable  Costs  (2006  $) 

Data  acquisition 

39,900 

34,100 

85,300 

160,000 

355,000 

Data  processing 

45,800 

35,200 

102,900 

250,000 

575,000 

Data  analysis  and  GIS  products 

94,300 

30,000 

68,100 

150,000 

220,000 

Data  reporting  and  documentation 

13,600 

8,500 

12,000 

15,000 

25,000 

Materials  and  consumables 

1,500 

1,000 

1,500 

5,000 

10,000 

Other  direct  costs 

0 

0 

0 

0 

0 

Subtotal  variable  costs 

195,100 

108,800 

269,800 

580,000 

1,185,000 

Total  project  cost  (2006  $) 

240,700 

151,100 

331,679 

665,000 

1,350,000 

Total  per/acre  cost  (2006  $) 

48.1 

26.8 

18.43 

13.3 

11.7 

Total  per/hectare  cost  (2006  $) 

125.8 

66.2 

45.53 

34.7 

30.7 
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5.2  COST  ANALYSIS 
5,2.1  Cost  drivers 

The  major  cost  drivers  for  the  two  demonstration  sites  were: 

•  Lidar  data  density  required.  For  the  Kirtland  site,  four  lidar  flights  were 
conducted;  two  concurrently  with  digital  imagery  collection,  and  two  lidar-only 
flights.  For  the  Victorville  site,  one  lidar/orthophoto  flight  and  one  lidar-only  flight 
were  conducted. 

•  Orthophoto  data  density  required.  For  the  Kirtland  site,  two  sets  of  digital  images 
were  collected,  and  orthophotos  were  created  at  10  cm  and  20  cm  pixel  sizes.  For  the 
Victorville  site,  only  10  cm  pixel  size  was  collected. 

•  Accuracy  and  precision  requirements.  A  higher  level  of  survey  control  was  needed 
at  the  Kirtland  PBR  site  than  for  production  sites,  in  order  to  verify  the  accuracy  and 
precision  of  the  lidar  and  orthophoto  data  and  to  confirm  that  data  could  be 
successfully  integrated  with  the  other  technologies  demonstrated.  It  is  estimated  that 
during  production  projects  fewer  survey  control  points  and  vertical  control  structures 
would  be  needed. 

•  Site  location/logistics.  The  Kirtland  PBR  site  location  affected  project  costs  both 
positively  and  negatively.  Positive  effects  were  that  the  Double  Eagle  Airport  was 
immediately  adjacent  to  the  project  site  and  the  helicopter  vendor  used  was  based  at 
this  site.  These  factors  lowered  mobilization  and  equipment  rental  costs  and 
eliminated  fees  for  fuel  transport.  Negative  factors  were  that  the  high  level  of  air 
traffic  at  the  airport  required  hiring  a  second  pilot.  In  addition  to  the  additional  labor 
cost,  the  additional  weight  resulted  in  higher  fuel  costs.  At  the  Victorville  site,  fuel 
had  to  be  transported  to  the  site  and  helicopter  flight  times  were  longer,  creating 
somewhat  higher  site-related  costs  than  for  Kirtland. 

In  addition  to  the  cost  drivers  listed  above,  costs  for  production  sites  will  be  affected  by  the 
following  additional  factors: 

•  Site  size.  Larger  sites  achieve  cost  savings  through  amortization  of  fixed  costs  such 
as  mobilization  and  project  planning,  as  well  as  through  increased  efficiency  in  data 
acquisition  and  processing. 

•  Vegetation  conditions.  Highly  vegetated  sites  will  have  higher  costs  due  to  the 
requirement  for  additional  lidar  passes  to  achieve  sufficient  density  of  points  reaching 
the  ground  surface. 
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•  Permitting  and  site  access  constraints.  DoD  sites  with  sensitive,  high-security 
areas  may  have  higher  costs.  However,  such  conditions  would  typically  affect  only 
pre-flight  planning  and  equipment  mobilization  costs  rather  than  data  acquisition, 
processing  and  analysis  costs.  Sites  with  environmental  constraints  do  not  normally 
impose  significantly  higher  costs  for  lidar  and  orthophotography,  since  the  airborne 
nature  of  the  technologies  does  not  typically  affect  sensitive  species  or  environments. 

5,2,2  Cost  Sensitivities  and  Additional  Potential  Savings 

Additional  savings  could  be  realized  through  either  of  the  following  methods: 

•  Acquiring  orthophotography  with  a  larger  pixel  size.  The  cost  of  acquiring  and 
processing  orthophotography  rises  dramatically  for  smaller  pixel  sizes,  and  acquiring 
orthophotos  at  20  cm  pixel  size  rather  than  10  cm  would  reduce  the  data  acquisition 
and  processing  costs  by  30  to  35%.  The  utility  of  such  photos  would  be  lower  since 
their  resolution  will  not  allow  discrimination  of  smaller  features.  Nevertheless,  this 
may  be  an  appropriate  solution  for  sites  where  orthophotography  is  inherently  less 
useful,  such  as  sites  with  heavy  tree  cover.  At  such  sites,  pre-existing 
orthophotography  from  other  sources  may  be  acceptable  if  its  positional  accuracy  can 
be  verified. 

•  Acquiring  lower-density  lidar  data.  Eliminating  the  assumed  second  lidar  flight, 
and  thus  only  collecting  lidar  with  the  10  cm  orthophoto  imagery,  would  reduce  costs 
by  25  to  30%.  The  ability  of  the  resulting  lidar  data  set  to  discriminate  features 
would  be  reduced;  however,  this  might  be  appropriate  if  the  lidar  data  was  to  be  used 
only  to  discriminate  large  features  such  as  bombing  targets  or  roads,  rather  than 
smaller  features  such  as  craters.  Alternatively,  DoD  could  specify  use  of  a  faster  lidar 
sensor,  which  could  meet  lidar  data  density  requirements  from  a  single  pass. 

Some  additional  cost  savings  could  potentially  be  achieved  by  establishing  Service-  or  DoD- 
wide  standards  for  data  acquisition,  GIS  data  product  creation,  data  delivery  formats,  and  project 
reporting. 


5.3  COST  COMPARISON 


Cost  comparisons  with  the  other  innovative  technologies  demonstrated  as  part  of  the  ESTCP 
WAA  Pilot  Program  will  be  made  in  the  Pinal  Report  for  the  WAA  Pilot  Program. 
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6.0  IMPLEMENTATION  ISSUES 

6.1  ENVIRONMENTAL  CHECKLIST 

No  environmental  regulations  applied  to  the  demonstration  and  no  permits  were  required. 


6.2  OTHER  REGULATORY  ISSUES 

Both  lidar  and  orthophotography  are  in  wide  eommereial  use.  Within  the  United  States,  no 
regulatory  restrietions  are  known  that  would  impede  the  wide  use  of  either  teehnology.  Outside 
of  the  United  States,  use  of  advaneed  IMU  equipment  may  be  restrieted  in  eertain  eountries.  The 
IMU  used  in  lidar  systems  is  military  dual  use  teehnology  and  international  use  requires  a  permit 
under  the  International  Traffie  in  Arms  Regulations  (22  CFR  120-130).  Additionally,  some 
eountries  impose  a  variety  of  restrietions  on  the  aequisition,  proeessing  and  subsequent  use  of 
lidar  and  orthophoto  data  eolleeted  within  their  borders,  partieularly  in  border  or  military-use 
areas.  Potential  users  of  lidar  in  sueh  situations  should  investigate  sueh  restrietions  as  part  of 
projeet  planning. 


6.3  END-USER  ISSUES 

Both  lidar  and  orthophotos  are  in  wide  eommereial  use  and  do  not  faee  substantial  end-user 


issues. 
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LIDAR  AND  ORTHOPHOTO  POSITIONAL  ACCURACY  RESULTS 

INTRODUCTION 

Verification  of  the  positional  accuracy  of  lidar  and  orthophoto  data  was  a  key  performance 
metric  for  the  demonstration.  To  be  useful  in  Wide  Area  Assessment,  lidar  and  orthophotos 
must  have  positional  accuracies  that  allow  integration  with  other  spatial  data  sets.  Lidar  and 
orthophotos  must  allow  for  meaningful  measurement  of  object  sizes  in  order  to  make 
conclusions  about  historic  munitions  use.  The  data  must  be  sufficiently  accurate  to  allow  field 
verification  of  their  results.  Quality  Assurance/Quality  Control  (QA/QC)  methods  for  lidar  and 
orthophotos  have  been  well  established,  and  standard  methods  were  applied  at  both  sites.  At 
both  sites,  primary  performance  metrics  for  both  technologies  were  met. 

DISCUSSION 

In  the  Demonstration  Plan,  performance  specifications  are  established  in  Table  4-2,  Performance 
Confirmation  Methods,  and  Section  4.3,  Data  Analysis,  Interpretation  and  Evaluation.  Positional 
confirmation  was  accomplished  through  comparison  of  lidar  and  orthophoto  positions  to 
surveyed  control  points,  vertical  control  structures,  and  comparison  of  lidar  and  orthophoto 
positions  to  each  other. 


RESULTS 

Surveyed  Control  Points 

Surveyed  control  points  formed  the  basis  for  analysis  of  positional  accuracy.  At  the  Kirtland 
site,  ESTCP  established  five  control  points  on  the  site;  TRSI  established  four  additional  control 
points  and  independently  occupied  two  of  the  ESTCP  control  points.  Both  control  point  sets 
were  referenced  to  NGS  Q424,  the  local  High  Accuracy  Reference  Network  (HARN)  point, 
located  approximately  6  km  south  of  the  study  area. 

At  the  Victorville  site,  TRSI  established  eight  survey  control  points  on  the  site.  These  were 
independently  occupied  by  ESTCP.  Control  point  locations  and  typical  control  points  are  shown 
in  Section  3.4,  and  are  available  in  the  project  QAQC  reports. 

Lidar  vertical  accuracies  -  Control  points 

Comparisons  were  made  between  the  elevations  of  the  control  points  and  the  lidar  points,  by 
comparing  the  surveyed  elevations  to  the  elevation  of  the  modeled  lidar  surface  at  that  point.  All 
values  were  within  the  target  accuracy  of  +/-  0.150  meters.  Values  for  the  Kirtland  site  were 
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calculated  using  both  ESTCP  and  TRSI  control  point  values.  Values  shown  in  the  ehart  below 
were  ealeulated  using  both  sets  of  survey  values.  At  the  Vietorville  site,  the  ESTCP  values  were 
not  used  to  ealculate  positional  accuraeies.  This  was  because  the  TRSI  and  ESTCP  surveyors 
had  a  vertieal  diserepaney  in  their  surveyed  values  for  the  control  points  of  approximately  1 
meter  (aeross  all  points)  whieh  eould  not  be  resolved.  This  diserepaney  was  probably  the  result 
of  differenees  in  survey  datums  used.  All  values  were  within  the  speeified  aeeuraey  standard. 

At  the  Kirtland  site,  results  followed  the  general  industry  expeetation  that  lidar  data  flown  at 
lower  elevations  will  be  more  aeeurate  than  at  higher  elevations,  however  the  eorrelation  was  not 
strong,  and  at  the  Victorville  site  the  aeeuraey  at  the  two  elevations  flown  was  essentially 
equivalent. 


Kirtland 

Lowest  Dz  (m) 

Highest  Dz  (m) 

Mean  Dz 

900  m  flight 

0.002 

0.333 

0.107 

450  m  flight 

0.043 

0.197 

0.113 

300  m  flight  1 

0.011 

0.227 

0.082 

300  m  flight  2 

0.012 

0.217 

0.087 

Victorville 

Lowest  Dz  (m) 

Highest  Dz  (m) 

Mean  Dz 

300  m  flight 

0.021 

0.333 

0.107 

450  m  flight 

0.021 

0.197 

0.113 

Lidar  Horizontal  Accuracy 


The  horizontal  aeeuraey  of  the  lidar  data  was  evaluated  by  eomparison  with  the  eontrol  points 
The  method  was  slightly  different  at  the  two  sites.  At  the  Kirtland  site,  the  eontrol  point 
positions  were  inferred  from  the  lidar  data  by  using  the  intensity  value  of  the  lidar  returns. 
Intensity  is  the  measured  value  of  the  strength  of  the  lidar  return  signal,  and  this  value  varies 
direetly  with  the  refleetivity  of  the  surfaee.  Eidar  points  that  returned  from  the  white  targets 
were  distinguishable  from  those  that  reflected  from  the  darker  colored  ground  surfaee.  These 
points  were  used  to  infer  the  position  of  the  interseetion  of  the  two  target  legs,  which  was  the 
loeation  of  the  eontrol  point.  This  loeation  was  then  eompared  to  the  surveyed  loeation  of  the 
measured  eontrol  point. 


At  the  Victorville  site,  the  lidar  points  refleeted  (by  ehanee)  from  the  antenna  of  the  GPS  base 
station  at  one  eontrol  point,  giving  an  exeellent  location  for  the  eontrol  point.  This  loeation  was 
used  for  the  eomparison  and  inferenee  of  the  other  eontrol  points  was  not  needed. 


The  method  is  illustrated  below. 
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Figure  A-1 

Method  for  Evaluating  Lidar  Horizontal  Accuracy 


Lidar  points  elassified  by  intensity,  blue  points  are 
refleetions  from  the  target  legs.  Kirtland 


* 


TAR3  DAY024 

Profde  view  of  lidar  points  showing  refleetion  from  the 
antenna  of  the  GPS  base  station  and  the  fiberglass  target 
legs. 


Orthophoto  image  with  lidar-derived  point 
added. 


Orthophoto  view  of  the  target  showing  the  same 
lidar  points. 


At  the  Kirtland  site,  the  average  x  and  y  displacement  (dx  and  dy)  for  all  targets  were  both  0.08 
m.  At  the  Victorville  site,  the  average  x  and  y  displacements  were  0.03  m  and  0.08  m  for  300  m 
lidar  flight  and  0.06  m  and  -0.03  m  for  the  450  m  flight.  All  values  are  well  within  the  0.65  m 
horizontal  accuracy  specification. 


Lidar  to  Orthophoto  Positional  Alignment 

Lidar  to  orthophoto  positional  alignment  was  evaluated  by  comparing  the  locations  of  features 
with  hard  edges  such  as  buildings  and  pavement  in  the  lidar  and  orthophoto  images.  The  edges 
of  buildings  and  pavement  were  located  visually  on  the  orthophotos.  In  the  lidar  data,  building 
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edges  were  loeated  by  examining  the  elevation  values  of  the  returns,  and  edges  of  pavement  were 
loeated  using  the  intensity  values.  The  proeess  is  illustrated  in  the  figures  below: 


Figure  A-2 

Method  for  Evaluating  Lidar  Positional  Alignment 


Kirtland  site,  Double  Eagle  Airport.  Buildings  in  lidar  are 
identified  by  elevation  values  of  the  lidar  returns. 


Vietorville  site,  vertieal  eontrol  strueture,  lidar  points 
elassified  by  elevation. 


Kirtland  site.  Double  Eagle  Airport  Runway. 
Pavement  edges  in  lidar  are  identified  by  intensity 
value. 


Vietorville  site,  old  RV,  lidar  points  classified  by 
elevation. 


At  the  Victorville  site,  this  calculation  was  more  difficult,  since  the  site  did  not  contain  buildings 
or  paved  roads.  The  vertical  control  structures  and  an  abandoned  RV  were  examined.  Results 
from  this  comparison  are  considered  less  reliable  than  for  the  Kirtland  site.  This  is  due  partially 
to  the  fact  that  the  vertical  control  structures  are  too  small  to  reliably  derive  a  corner  location 
from  the  lidar  data  alone.  (The  example  shown  is  the  best  case). 


Results  are  shown  for  the  two  sites: 
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Specification  (m) 

Average  Dx  (m) 

Average  Dy  (m) 

Kirtland  20  em  pixels 

0.40 

0.167 

0.360 

Kirtland  10  em  pixels 

0.20 

0.137 

0.167 

Vietorville  10  em  pixels 

0.20 

0.10 

0.10 

Orthophoto  Positional  Accuracy 

Orthophoto  horizontal  accuracy  was  evaluated  by  deriving  the  location  of  the  surveyed  control 
points  in  the  orthophoto  images,  and  then  comparing  these  to  their  surveyed  locations.  The 
method  is  illustrated  in  the  figure  below. 


Figure  A-3 

Orthophoto  Comparison  to  Control  Points 


The  red  star  represents  the  eontrol  point  loeation  in  the 
orthophoto;  the  blue  star  is  its  surveyed  loeation.  The 
points  are  approximately  1  pixel  width  (10  em)  apart. 

The  established  accuracy  specification  was  2  pixel  widths;  all  orthophoto  data  met  this 
specification,  as  shown  below. 


Specification  (m) 

Maximnm  offset  (m) 

Mean  offset  (m) 

Kirtland  20  cm  pixels 

0.60 

0.561 

0.210 

Kirtland  10  cm  pixels 

0.30 

0.279 

0.146 

Victorville  10  cm  pixels 

0.30 

0.143 

0.072 
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CONCLUSIONS 

Lidar  and  orthophoto  data  at  both  sites  met  the  established  positional  aecuracy  specifieations. 


APPENDIX  B 

Combining  Lidar  Data  from  Multiple  Flights 
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COMBINING  LIDAR  DATA  FROM  MULTIPLE  FLIGHTS 

INTRODUCTION 

When  lidar  data  from  different  flights  at  the  Kirtland  site  was  eombined,  the  resulting  surfaee 
models  showed  serious  degradation.  Features  that  were  clearly  shown  in  the  data  from 
individual  flights  were  obscured  in  the  combined  data.  This  effect  was  particularly  pronounced 
when  flights  from  different  altitudes  were  combined.  The  cause  of  this  effect  has  implications 
for  the  acquisition  of  lidar  data,  including  appropriate  contract  specifications. 


DISCUSSION 

All  lidar  data  acquisition  contracts  specify  the  vertical  datum  of  the  output  data;  normally  the 
datum  specified  is  the  North  American  Vertical  Datum  of  1988  (NAVD  88).  In  the  case  of  the 
Kirtland  site,  the  lidar  vendor  was  requested  to  deliver  each  of  the  four  flights  as  a  separate  data 
set,  each  in  NAVD  88.  This  specification  was  met. 

However,  this  contract  specifications,  with  adjustment  of  each  individual  flight  to  the  contract 
vertical  datum,  is  not  standard  industry  practice.  Rather,  standard  practice  is  to  choose  only  one 
of  the  flights  which  will  be  calibrated  to  the  specified  vertical  datum,  after  which  data  from  the 
rest  of  the  flights  is  calibrated,  not  to  the  vertical  datum,  but  to  data  from  the  first  flight.  It  is  also 
more  standard  practice  to  deliver  only  one,  combined  data  set.  The  contract  specification  used 
for  the  Kirtland  site  resulted  from  the  decision  to  deliver  the  lidar  at  each  elevation  as  a  separate 
date  file.  Delivery  of  the  Demonstration  Plan  appeared  to  be  the  most  conservative  approach  and 
to  introduce  the  fewest  additional  variables.  In  fact,  this  approach  illustrated  the  wisdom  of  the 
standard  industry  practice. 

The  Kirtland  data  illustrates  the  results  of  specifying  that  each  flight  be  adjusted  only  to  the 
project  vertical  datum.  Despite  the  fact  that  each  flight’s  data  set  was  correctly  calibrated  to  the 
vertical  datum,  and  each  was  well  within  contract  specifications,  small  errors  in  the  equipment 
lead  to  each  flight  being  slightly  out  of  alignment  with  the  next.  These  slight  misalignments 
create  a  “blurring”  of  the  resulting  ground  surface  in  the  combined  data  set. 
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Figure  B-1 

Kirtland  Individual  and  Combined  Lidar  Data  As  Delivered 


300  m  lidar,  single  flight. 

Combined  900  m  and  450  m  lidar 

Combined  lidar,  all  flights. 

flights. 

Figure  B-2 

Kirtland  Combined  Lidar  Data  Following  Reprocessing 

As  a  test,  a  small  sample  of  the  Kirtland  data  was  reproeessed  to  aehieve  the  best  fit  possible 
between  the  two  data  sets.  Results  were  promising,  with  the  reproeessed  data  yielding  a  mueh 
more  useable  surfaee  model. 


Combined  900  and  450  m  lidar  flights.  Combined  900  and  450  m  lidar,  following  adjustment  of 

the  two  flights  for  best  fit. 


At  the  Vietorville  site,  eontraet  speeifieations  were  ehanged.  The  two  flights  were  delivered  as 
separate  data  sets,  as  with  the  Kirtland  data.  However,  the  900  m  flight  was  ealibrated  to  NAVD 
88,  and  the  450  m  flight  was  adjusted  to  the  best  possible  fit  with  the  900  m  data  set.  As  a  result, 
the  450  m  data  was  not  truly  in  NAVD  88.  However,  the  two  data  sets  eould  then  be  eombined 
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to  create  a  higher-density  eombined  data  set  that  eould  be  used  to  examine  the  effeets  of  higher 
overall  point  density. 


Figure  B-3 

Victorville  Individual  and  Combined  Lidar  Data 
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300  m  lidar  flight.  Combined  300  m  and  450  m  flights. 


CONCLUSIONS 

In  projects  where  multiple  lidar  flights  will  be  eondueted,  aequisition  speeifieations  should  be 
written  to  recognize  current  industry  practice  for  combining  lidar  data  sets.  Contracts  should  not 
specify  that  each  individual  flight  should  be  independently  calibrated  to  the  specified  vertical 
datum.  Rather,  the  vertical  datum  requirement  should  be  applied  to  data  from  one  flight,  with 
subsequent  data  sets  adjusted  to  the  first. 


APPENDIX  C 

Variation  in  Lidar  Data  Density  and  Potential  Effect  on  Feature  Identification 
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VARIATION  IN  LIDAR  DATA  DENSITY  AND  POTENTIAL  EFFECT  ON  FEATURE 

DETECTION 


INTRODUCTION 

Lidar  data  density  is  generally  expressed  as  either  points  per  square  meter  or  average  spaeing 
between  lidar  points,  averaged  over  the  entire  site.  In  praetiee,  lidar  point  density  was  found  to 
vary  eonsiderably  over  the  site  depending  on  faetors  sueh  as  flight  line  overlap  and  wind 
eonditions.  These  variations  probably  do  not  affeet  the  ability  of  lidar  to  deteet  large  features 
sueh  as  bombing  targets,  but  may  affeet  the  ability  to  deteet  smaller  objeets,  partieularly  those  at 
or  below  1  meter  in  size.  Measures  of  overall  density  remain  useful  as  general  indieators  of  the 
effeetiveness  of  lidar.  However,  data  density  speeifieations  may  need  to  be  adjusted  where 
deteeting  small  objeets  is  a  priority. 


DISCUSSION 

Lidar  data  density  is  affeeted  by  at  least  five  operational  faetors,  ineluding  the  following: 

•  flight  altitude, 

•  flight  speed, 

•  flight  line  overlap, 

•  laser  pulse  rate,  and 

•  mirror  rotation  speed. 

Data  density  is  also  affeeted  by  at  least  three  site  faetors: 

density  of  vegetation  eover, 

steepness  of  terrain,  and 

wind  speed  during  data  eolleetion. 

Finally,  the  density  of  lidar  points  used  to  ereate  the  ground  surfaee  model  is  very  substantially 
affeeted  by  how  lidar  points  are  elassified  as  ground  or  non-ground  refleetions. 

At  both  demonstration  sites,  the  intention  of  the  demonstration  was  to  vary  only  flight  altitude,  as 
a  means  to  vary  the  overall  lidar  point  density.  Other  parameters  were  held  as  eonstant  as 
possible,  ineluding  flight  speed,  flight  line  overlap,  laser  pulse  rate  and  mirror  rotation  speed. 
Slopes  on  the  site  did  not  vary  suffieiently  to  affeet  data  density.  Vegetation  on  the  site  was 
relatively  sparse  and  also  relatively  eonstant  throughout  the  site,  and  it  was  assumed  that 
vegetation  elassifieation  rates  would  be  relatively  eonstant. 
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Nevertheless,  flight  line  overlap  and  vegetation  elassifieation  had  a  very  significant  impact  on 
lidar  point  density.  In  some  cases  these  factors  had  a  much  larger  impacts  than  flight  altitude  in 
determining  lidar  data  density. 

Lidar  data  densities  achieved  for  the  four  Kirtland  flights  and  two  Victorville  flights  are  given  in 
the  table  below.  These  are  overall  densities,  derived  by  dividing  the  number  of  lidar  points  by 
the  area  of  the  site,  for  the  entire  lidar  point  data  set  before  classification  as  ground  and  non¬ 
ground  returns. 


Table  C-1 

Overall  Lidar  Data  Densities 


Demonstration  Site 

Flight 

Point  density  (pts/m^) 

Kirtland 

900  m  North  Block 

1.4 

900  m  South  Block 

1.6 

450  m  north  block 

5.2 

450  m  south  block 

4.1 

300  m  east-west  north  block 

5.2 

300  m  north-south  north  block 

6.5 

300  m  flight  1  south  block 

5.9 

300  m  flight  2  south  block 

6.1 

Victorville 

450  m  flight 

4.8 

300  m  flight 

6.4 

When  examined  more  closely,  lidar  point  density  was  found  to  vary  considerably  within  each 
flight.  The  following  figures  give  a  graphical  illustration  of  some  of  the  ways  that  lidar  point 
density  varied.  In  all  of  the  following  figures,  darker  colors  indicate  higher  point  density,  and 
the  fine  blue  lines  indicate  the  lidar  flight  lines. 
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Figure  C-1 

Kirtland  Site  Lidar  Point  Density  Maps  -  All  Lidar  Points 
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300  m  south  bloek.  Flight  1,  all  points. 
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Points  p«r  squaro  meter 
I  0  -o.e 
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Points  per  square  meter 
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10.9*126 


300  m  north  bloek  North-South  flight,  all  points. 


300  m  south  bloek,  Flight  2,  all  points. 


The  images  above  are  for  all  lidar  points  regardless  of  classification  as  ground  or  non-ground 
reflections.  Once  the  points  classified  as  vegetation  are  removed,  both  overall  point  density  and 
the  variability  of  the  point  density  is  reduced,  as  shown  in  the  following  figures. 


Figure  C-2 

Kirtland  Site  Lidar  Point  Density  Maps  -  Ground  Returns  Only 


900  m  north  block,  ground  points.  900  m  south  block,  ground  points. 
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300  m  north  bloek  EW,  ground  points. 


300  m  south  bloek,  Flight  1,  ground  points. 
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300  m  north  bloek  NS,  ground  points.  300  m  south  bloek  F2,  ground  points. 


The  following  figures  present  elose-up  views  of  the  point  density  maps  for  one  of  the  300  meter 
lidar  data  sets,  ineluding  images  for  both  all  lidar  points  and  ground  points  only. 

These  results  illustrate  one  limitation  of  measuring  lidar  point  density  for  the  site  as  a  whole; 
some  areas  of  the  site  have  a  higher  point  density  than  speeified,  while  other  areas  have  a  lower 
density. 


Figure  C-3 

Lidar  Point  Density  Maps  -  Detailed  Views 


450  m  all  points,  scale:  1-400. 
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300  m  east-west,  ground  points,  seale:  1-400. 

Similar  results  were  obtained  for  the  Victorville  site,  as  shown  below: 


Figure  C-4 

Victorville  Site  Data  Density  Maps 


300  m  all  points. 


300  m  ground  points  only. 
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450  m  all  points. 


450  m  ground  points  only. 


All  of  these  figures  show  that  lidar  point  density  in  the  overall  lidar  data  set  is  highly  influenced 
by  flight  line  overlap.  Point  density  also  varies  along  the  flight  line,  with  bands  of  higher  and 
lower  point  density.  These  bands  are  the  result  of  flight  conditions  such  as  wind  gusts  or  sudden 
changes  in  pitch  of  the  aircraft,  which  result  in  wider  or  narrower  spacing  between  the  lines  of 
lidar  points.  This  effect  is  particularly  pronounced  in  the  all  points  data  set. 


DATA  DENSITY  AND  POINT  CLASSIFICATION 

Data  density  variation  appears  to  fall  significantly  as  vegetation  is  removed,  even  for  these  two 
relatively  sparsely-vegetated  sites.  This  result  was  unexpected;  vegetation  conditions  were 
relatively  uniform  on  both  sites,  so  variation  in  data  density  caused  by  flight  line  overlap  should 
theoretically  be  the  same  for  either  all  points  or  ground  points.  This  effect  results  from  the 
methods  used  to  classify  points. 

Classification  of  lidar  points  as  ground  or  non-ground  is  performed  using  automated  algorithms 
followed  by  operator  inspection  and  editing.  Standard  point  classification  methods  are  aimed  at 
creating  an  accurate  ground  surface  model  free  of  vegetation  artifacts.  As  such,  they  are  biased 
towards  vegetation  removal.  The  following  table  shows  the  percentages  of  points  classified  as 
ground  and  non-ground  for  the  four  lidar  flights  at  the  Kirtland  demonstration  site. 
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Table  C-2 

Percentages,  Ground  and  Non-Ground  Lidar  Returns 


Lidar  Flight 

Total  Points 

Ground 

Points 

Ground  % 

Non-Ground 

Points 

Non- 

Ground  % 

900  m 

31,212,625 

22,503,996 

72.1% 

8,708,629 

27.9% 

450  n 

102,136,027 

42,849,155 

42.0% 

59,286,872 

58.0% 

300  m  NS  and  SB  F2 

110,320,328 

42,863,611 

38.9% 

67,456,717 

61.1% 

300  m  EW  and  SB  FI 

129,953,682 

45,906,149 

35.3% 

84,047,533 

64.7% 

This  table  shows  that  as  the  number  of  overall  lidar  points  inereased,  the  pereentage  classified  as 
ground  points  decreased.  That  is,  part  of  the  increased  overall  data  density  was  lost  from  the 
ground  surface  model.  This  finding  is  unexpected  since  vegetation  was  identical  for  each  flight. 
In  order  to  examine  this  phenomenon  in  more  detail,  ten  sample  blocks  were  selected,  each  25  x 
25  m,  covering  a  range  of  data  densities.  The  sample  blocks  are  shown  in  the  following  figure. 

Figure  C-5 

Data  Density  Analysis  Blocks 


Point  density  test  blocks  with  300  m  lidar  point  density 


Point  density  test  blocks  with  450  m  lidar  point  density 
map. 


map. 

For  each  sample  block,  the  total  number  of  lidar  points  was  calculated,  along  with  the  percentage 
of  ground  and  non-ground  returns.  Similar  calculations  were  performed  for  each  of  the  four  lidar 
flights.  The  data  showed  a  strong  correlation  between  overall  data  density  (total  number  of 
points)  and  percentage  of  points  classified  as  non-ground.  As  the  total  number  of  points 
increased,  the  percentage  of  non-ground  points  rose  from  around  30%  to  around  80%. 
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Figure  C-6 

Lidar  Point  Classification  vs.  Point  Density 


900  m  LiDAR  -  Total  Points  vs  %  Non-ground 


450  m  LIDAR  -  Total  Points  vs  %Non-ground 
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This  result  is  significant  since  acquiring  more  lidar  density  raises  the  project  costs.  These  results 
appear  to  indicate  that  at  least  a  significant  portion  of  the  additional  lidar  points  being  acquired  is 
not  reaching  the  ground  surface  models. 

The  most  likely  explanation  for  this  effect  is  based  on  the  fact  that,  as  shown  in  the  point  density 
maps,  the  highest  areas  of  point  density  occur  where  flight  lines  overlap.  In  these  areas,  lidar 
points  from  the  different  flight  lines  are  always  very  slightly  misaligned,  (even  when  the  data  is 
well  within  the  vertical  accuracy  specification),  with  one  data  set  slightly  below  and  the  other 
slightly  above  each  other.  This  causes  the  surface  in  this  overlap  area  to  be  “rough”,  and  it  is  this 
roughness  that  the  automated  point  classification  algorithm  uses  as  an  indicator  of  non-ground 
points  such  as  low  vegetation.  This  phenomenon  deserves  further  investigation. 
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LIDAR  DATA  DENSITY  DISTRIBUTION  AND  FEATURE  DETECTION 

Detecting  MEC  features  using  lidar  requires  that  the  lidar  points  be  dense  enough  to  reflect  from 
both  the  feature  and  its  surroundings.  Variations  in  lidar  point  density  should  affect  the  size  of 
features  that  can  reliably  be  detected. 

To  examine  this  variation  more  closely,  the  lidar  density  maps  shown  above  were  queried  to 
determine  the  number  of  lidar  points  in  each  square  meter  of  the  study  area.  The  analysis  was 
performed  for  both  the  entire  point  set  and  for  ground  returns  only.  Results  were  as  follows: 


Table  C-3 

Kirtland  lidar  Data  Density  Distribution  -  All  Points 


All  values  are  in  points  per  square  meter. 

Flight 

Min 

Max 

Mean 

Standard 

Deviation 

% 

Value  =  0 

900  m  all  points 

0 

58 

1.43 

0.89 

24.28 

450  m  all  points 

0 

150 

4.68 

2.74 

0.19 

300  m  north  block  east-west  all  points 

0 

118 

6.22 

3.25 

0.14 

300  m  north  block  north-south  all  points 

0 

73 

4.98 

2.57 

0.36 

300  m  south  block  flight  1  all  points 

0 

158 

5.34 

3.20 

0.12 

300  m  south  block  flight  2  all  points 

0 

126 

5.49 

3.70 

1.26 

Table  C-4 

Kirtland  lidar  Data  Density  Distribution  -  Ground  Points  Only 


All  values  are  in  points  per  square  meter. 

Flight 

Min 

Max 

Mean 

Standard 

Deviation 

% 

Value  =  0 

900  m  ground  points 

0 

12 

1.03 

0.72 

38.46 

450  m  ground  points 

0 

17 

1.98 

1.23 

11.10 

300  m  north  block  east-west  ground  points 

0 

14 

2.21 

1.38 

10.20 

300  m  north  block  north-south  ground 
points 

0 

14 

2.03 

1.38 

12.59 

300  m  south  block  flight  1  ground  points 

0 

17 

1.86 

1.32 

14.51 

300  m  north  block  flight  2  ground  points 

0 

12 

1.83 

1.35 

16.62 

Table  C-5 

Victorville  lidar  Data  Density  Distribution  -  All  Points 


All  values  are  in  points  per  square  meter. 

Flight 

Min 

Max 

Mean 

standard 

Deviation 

% 

Value  =  0 

450  m  all  points 

0 

346 

msm 

300  m  all  points 

0 

220 

2.96 

1.54% 
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Table  C-6 

Victorville  lidar  Data  Density  Distribution  -  Ground  Points  Only 


All  values  are  in  points  per  square  meter. 

Flight 

Min 

Max 

Mean 

Standard 

Deviation 

% 

Valne  =  0 

450  m  ground  points 

0 

42 

1.91 

1.21 

10.87% 

300  m  ground  points 

0 

28 

2.59 

1.56 

7.89% 

These  tables  illustrate  that  even  at  the  highest  overall  data  densities  aehieved;  there  are 
nevertheless  one-meter  grid  eells  where  there  are  no  lidar  points.  For  ground  points  -  the  data 
most  typieally  used  to  ereate  surfaee  models  -  the  pereentage  of  one-meter  eells  with  no  lidar 
points  ranges  from  7.89%  to  38.46%.  This  is  a  potentially  signifieant  finding  sinee  one  use  of 
lidar  is  to  detect  small  features  in  order  to  delineate  MRS  boundaries.  An  examination  of  the 
actual  lidar  point  patterns  on  the  ground  demonstrates  that  in  some  circumstances,  objects  in  the 
1 .0  -  1 .5  m  range  could  easily  be  missed  in  the  900  m  flight. 

The  following  figures  show  the  Kirtland  test  crater  area  with  surface  models  created  from  the 
various  lidar  data  sets.  Surfaces  were  created  using  both  the  full  lidar  data  set  and  the  points 
classified  as  ground  reflections.  This  approach  was  taken  in  order  to  determine  whether 
vegetation  removal  significantly  affected  the  ability  of  the  lidar  data  to  distinguish  the  craters. 
Darker  green  colors  represent  lower  elevations. 
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Figure  C-7 

Test  Craters  and  lidar  Data  Density 


FINAL  REPORT 

High  Density  Lidar  and  Orthophotography 
in  UXO  Wide  Area  Assessment 
Contraet  No.  N44255-02-D-2008 


Appendix  C 
Revision  No.:  0 
Date:  August  2007 
Page  C-14 


300  m  lidar  flight  2,  ground  points  only. 


300  m  lidar  flight  2,  ground  (blaek)  and  non-ground 
(blue)  points. 


300  m  lidar  flight  1,  ground  points  only. 


300  m  lidar  flight  1,  ground  (blaek)  and  non-ground 


These  figures  illustrate  several  points: 


•  One  of  the  1 .0  meter  craters  was  missed  completely  using  ground  returns  from  both  300 
m  flights,  despite  the  fact  that  these  were  the  flights  with  the  highest  overall  point 
densities. 


Data  density  in  the  test  crater  area  was  higher  for  the  450  m  flight  than  for  either  300  m 
flight,  even  though  overall  data  density  for  the  450  m  flight  was  lower. 
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•  The  1.5  m  craters  were  detected  with  roughly  equal  success,  at  all  of  the  data  densities 
tested.  Nevertheless,  there  are  areas  in  the  900  m  and  both  300  m  ground  point  sets 
where  a  1.5  m  crater  would  not  receive  any  lidar  points. 


CONCLUSIONS 

Measures  of  overall  lidar  density  such  as  points  per  square  meter  or  average  distance  between 
points  do  not  capture  the  actual  variations  of  data  density  observed  at  the  demonstration  sites. 
Observed  variation  resulted  primarily  from  flight  line  overlap  and  wind  conditions,  and  was 
often  a  more  important  factor  than  flight  altitude  in  affecting  density  for  a  particular  area  of  the 
site. 

Point  classification  methods  had  a  strong  effect  on  lidar  density  of  the  ground  surface  model.  In 
effect,  much  of  the  extra  point  density  achieved  in  the  flight  line  overlap  areas  was  removed  by 
the  point  classification  methods  used. 

Variations  in  lidar  density  did  not  affect  the  detection  of  the  1.5  m  test  craters  at  the  Kirtland  site, 
but  three  of  the  four  lidar  data  sets  nevertheless  contained  areas  where  a  crater  of  this  size  could 
be  missed.  Detection  of  the  1.0  m  test  craters  was  affected  by  lidar  density  variations,  and  the 
0.32  m  craters  were  not  detected  at  any  of  the  densities  tested. 


APPENDIX  D 

GIS-Based  Methods  for  Creating  Ground  Surface  Models  from  Lidar  Points 
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GIS-BASED  METHODS  FOR  CREATING  GROUND  SURFACE  MODELS  FROM 

LIDAR  POINTS 


INTRODUCTION 

Surface  models  ean  be  ereated  from  lidar  points  in  a  variety  of  ways.  All  involve  some  level  of 
interpolation  from  the  random  lidar  points  to  a  regularly-spaced  array  that  ean  be  analyzed  using 
GIS  methods.  A  limited  eomparison  of  some  standard  methods  of  ereating  surfaee  models 
showed  that  the  methods  used  to  proeess  the  demonstration  data  gave  the  clearest  definition  of 
the  Kirtland  area  test  eraters.  However,  the  difference  was  not  large,  and  the  eraters  were  visible 
using  most  of  the  methods  tested. 


DISCUSSION 

There  are  two  general  approaehes  to  creation  of  surfaee  models,  all  result  in  a  regularly-spaeed 
grid  with  eaeh  eell  assigned  an  elevation  value. 

The  first  approaeh  is  to  interpolate  the  surfaee  model  direetly  from  the  lidar  points.  There  are  a 
variety  of  mathematieal  approaehes  to  sueh  interpolation.  These  inelude  the  following; 

•  Inverse  Distance  Weighting  (IDW)  interpolation  determines  cell  values  using  a 
linearly  weighted  eombination  of  a  set  of  sample  points.  The  weight  is  a  function  of 
inverse  distanee,  and  the  surfaee  being  interpolated  should  be  that  of  a  loeationally 
dependent  variable.  IDW  interpolation  allows  the  user  to  eontrol  the  effeet  of  known 
points  on  the  interpolated  values,  based  upon  their  distance  from  the  output  point. 

The  eharaeteristies  of  the  interpolated  surfaee  ean  also  be  eontrolled  by  limiting  the 
input  points  used  to  ealeulate  eaeh  interpolated  point.  The  input  ean  be  limited  by  the 
number  of  sample  points  to  be  used,  or  by  a  radius  within  whieh  there  are  all  points  to 
be  used  in  the  ealculation  of  the  interpolated  points. 

•  Kriging  is  an  advaneed  geostatistieal  proeedure  that  generates  an  estimated  surfaee 
from  a  seattered  set  of  points  with  z  values.  Kriging  is  based  on  a  regionalized 
variable  theory  that  assumes  that  the  spatial  variation  in  the  phenomenon  represented 
by  the  z  values  is  statistieally  homogeneous  throughout  the  surfaee  (i.e.,  the  same 
pattern  of  variation  ean  be  observed  at  all  loeations  on  the  surfaee).  This  hypothesis 
of  spatial  homogeneity  is  fundamental  to  the  regionalized  variable  theory. 

•  Splining  performs  a  two-dimensional  minimum  curvature  spline  interpolation  on  a 
point  data  set  resulting  in  a  smooth  surfaee  that  passes  exaetly  through  the  input 
points.  This  basie  minimum-curvature  technique  is  also  referred  to  as  thin  plate 
interpolation.  It  ensures  a  smooth  (eontinuous  and  differentiable)  surfaee  together 


FINAL  REPORT 

High  Density  Lidar  and  Orthophotography 
in  UXO  Wide  Area  Assessment 
Contraet  No.  N44255-02-D-2008 


Appendix  D 
Revision  No.:  0 
Date:  August  2007 
Page  D-2 


with  continuous  first-derivative  surfaees.  Rapid  changes  in  gradient  or  slope  (the  first 
derivative)  may  oecur  in  the  vicinity  of  the  data  points;  hence  this  model  is  not 
suitable  for  estimating  second  derivative  (curvature). 

The  seeond  approaeh  is  to  ereate  a  Triangualted  Irregular  Network  (TIN)  using  every  individual 
lidar  point.  The  TIN  is  then  used  to  ereate  the  surfaee  model,  again  using  one  of  the 
interpolation  methods  described  above.  Either  approach  can  be  employed  using  the  ground 
points  only  or  both  the  ground  and  vegetation  points. 

Both  of  the  above  approaehes  produee  good  surfaces,  with  eaeh  method  having  its  own  particular 
advantages  and  shortcomings.  Every  method  uses  one  teehnique  or  other  to  interpolate  (or  guess 
at)  the  surfaee  between  the  lidar  points.  However,  by  creating  a  TIN  from  the  points  before 
creating  the  grid  surface  model,  we  assure  that  each  individual  point  is  represented  aecurately  in 
the  final  product,  and  in  the  TIN  itself  which  can  be  examined  in  partieular  eases.  Consequently, 
for  this  demonstration,  the  TIN  method  was  used  for  all  data  sets. 

A  test  of  these  alternative  methods  was  conducted  using  the  data  from  the  Kirtland  site.  The 
points  in  the  area  surrounding  the  test  eraters  was  extracted  and  used  to  ereate  a  series  of  surfaces 
using  the  available  methods. 
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Figure  D-1 

Surface  Model  Examples 


10  m  pixel  orthophoto. 


900  m  TIN. 


900  m  TIN-derived  IDW. 


900  m  spline. 
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450  m  TIN-derived  IDW. 


450  m  spline. 
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450  m  IDW. 


450  m  kriging. 


These  figures  illustrate  that  the  TIN-derived  surface  models  give  a  slightly  more  clear 
representation  of  the  1.5  and  1.0  m  test  craters.  However,  all  methods  showed  the  presence  of 
these  four  test  craters. 


CONCLUSIONS 

Lidar  surface  interpolation  methods  tested  appeared  to  have  a  weak  effect  on  feature  detection, 
with  the  TIN-to-surface  model  method  used  to  process  the  demonstration  data  yielding  the  best 
results.  However,  the  difference  between  the  methods  was  not  pronounced. 


APPENDIX  E 

Automation  of  Lidar  Data  Processing  in  the  ESRI  GIS  Environment 
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AUTOMATION  OF  LIDAR  DATA  PROCESSING  IN  THE  ESRI  GIS  ENVIRONMENT 

INTRODUCTION 

Many  standard  lidar  data  processing  steps  ean  be  suceessfully  automated.  This  is  important 
sinee  lidar  data  sets  are  very  large,  often  involving  many  hundreds  of  data  files,  all  of  whieh  are 
proeessed  using  essentially  the  same  steps.  Automation,  when  sueeessful,  ean  reduee  eosts  and 
inerease  the  reliability  of  the  resulting  produets.  For  this  demonstration,  all  automation  was 
aecomplished  using  ESRFs  AreGIS  program.  This  software  was  chosen  beeause  it  is  in  wide 
use  by  federal  land  managers  who  may  use  lidar  data. 


DISCUSSION 

AreGIS  provides  several  avenues  for  proeess  automation,  ineluding  the  following; 

•  Arc  Macro  Language  (AML).  AML  is  a  scripting  language  developed  for  the 
Areinfo  Workstation  environment.  It  is  used  to  automate  data  proeessing  and 
analysis  tasks,  the  ereation  of  eomplex  maps,  and  to  develop  custom  interfaces  to  aid 
user  with  various  tasks  sueh  as  data  creation,  editing  and  analysis.  .It  is  a  robust,  full- 
featured  programming  language  that  can  be  used  to  automate  the  commands  that  can 
be  used  at  any  of  the  Areinfo  eommand  line  prompts  (i.e.,  ARC:,  ARCLDIT;, 
ARCPLOT;).Its  main  disadvantage  is  that  any  non  3D  spatial  data  proeessing  must  be 
done  using  the  antiquated  Areinfo  eoverage  format.  Only  in  some  very  limited 
situations  can  Areinfo  shapefiles  be  used,  and  AreGIS  feature  elasses  and 
geodatabases  eannot  be  used  at  all.  Another  disadvantage  is  that  the  very  steep 
learning  eurve  required  to  beeome  profieient  in  AML.  Lven  though  it  is  based  on 
standard  programming  eoneepts,  AML  is  a  proprietary  language  unique  to  Areinfo 
Workstation  and  is  not  easy  to  learn. 

•  Visual  Basie.  LSRI  has  written  a  library  of  programming  objeets  ealled  AreObjeets 
that  allows  the  programmer  to  develop  powerful  applications  relatively  quiekly  using 
the  very  popular  Visual  Basie  for  Applieations  (VBA)  programming  language  that 
comes  bundled  with  AreGIS.  AreObjeets  ean  also  be  used  with  the  Visual  Studio 
.NET  languages,  VB.NET,  C++ .NET  and  C#.NET.  Applieations  written  in  VBA 
must  be  run  from  an  AreGIS  applieation  like  ArcMap,  AreCatalog  or  AreSeene  while 
Visual  Studio.NET  applieations  ean  be  run  independently  of  AreGIS  though  access  to 
the  AreGIS  lieense  server  is  still  required. 

•  Python.  Python  is  an  open  source,  objeet  oriented,  programming  language  developed 
to  handle  the  kind  of  tasks  normally  given  to  seripting  languages  like  PERL,  Ruby  or 
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Tcl  but  without  the  steep  learning  eurve  usually  assoeiated  with  many  powerful 
objeet  oriented  languages.  ESRI  has  written  its  Geoproeessing  objeet  speeifieally  to 
be  used  with  Python.  Python  is  a  relatively  easy  language  to  learn,  and  ESRI  also 
provides  some  pre-designed  user  interfaees  that  eustom  Python  seripts  ean  be  linked 
to.  Unfortunately,  ESRI  has  made  only  the  Geoproeessing  objeet,  and  not  all  of  its 
AreObjeets  eolleetion,  available  to  Python  developers.  In  order  to  use  ESRI’s  more 
extensive  AreObjeets  library,  the  developer  must  use  VBA  or  one  of  the  .NET 
programming  languages. 

Eor  this  demonstration,  automation  was  earried  out  using  Python.  The  deeision  to  use  Python 
was  based  on  its  ease  of  use,  combined  with  the  need  to  develop  the  automation  scripts  quickly 
and  start  the  processing  of  the  raw  lidar  data  as  soon  as  possible.  AML  scripts  had  already  been 
developed  to  automate  lidar  data  processing,  however  the  conversion  to  Python  eliminated  the 
need  to  convert  the  lidar  data  sets  to  coverages.  Python  was  used  to  create  a  suite  of  tools  that 
handles  the  processing  of  the  lidar  files  delivered  by  the  vendor,  starting  from  the  creation  of  the 
initial  point  shapefiles  and  continuing  through  the  final  surface  models  and  data  analysis.  The 
processing  scripts  were  designed  to  handle  batch  processing  of  input  data  fdes,  which  required 
that  a  consistent  naming  convention  be  established  and  followed  input  data. 

While  Python  was  the  approach  taken,  automation  results  and  process  steps  would  have  been 
essentially  the  same  regardless  of  the  scripting  approach  chosen. 

1 .  The  first  step  was  to  create  point  shapefiles  from  the  ASCII  files  containing  the  lidar 
points  in  ArcGIS. 

2.  The  second  step  was  to  generate  a  continuous  surface  model  TINs  using  each  point 
shapefile  as  an  input. 

3.  The  third  step  was  to  create  DEMs,  or  surface  model  grids,  from  each  of  the  TINs. 
Eor  both  Victorville  and  Kirtland,  a  1  foot  cell  size  was  used  for  the  DEMs  as  the 
highest  resolution  the  lidar  data  would  support. 

These  three  steps  were  run  as  separate  scripts.  There  would  be  no  barrier  to  combining  all  three 
processes  into  one  script.  However,  running  each  script  separately  allowed  for  QA/QC 
inspection  between  steps.  This  was  significant  since  each  processing  step  was  relatively  time- 
consuming,  and  it  was  important  to  catch  any  errors  in  the  data  at  the  earliest  point  possible. 


APPENDIX  F 
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ANALYTICAL  METHODS  SUPPORTING  THE  EXPERIMENTAL  DESIGN 

Analytical  methods  appropriate  to  lidar  and  orthophoto  colleetion  include  calculations  of  target 
lidar  point  density,  orthophoto  pixel  size,  and  along-flight-line  swath  width.  Formulas  for  these 
calculations  are  given  below. 

Lidar  point  density.  Point  density  is  calculated  using  the  following  formula: 

Pt  Density  =  Pulse  Rate  /  (Forward  Flying  Speed  *  Swath  Width) 

=  50  000  /  (20  *  204) 

=  12.25  shots/m 

Pixel  size.  Pixel  size  is  ealeulated  using  the  following  formula: 

Pixel  size  =  (Altitude  *  CCD  Grid  Size)  /  Focal  Length 
=  (700  (m)  *  0.000008  (m))  /  0.035  (m) 

=  0.16  m@ 

Swath  width.  The  swath  width  is  calculated  using  flying  height  and  maximum  mirror  angle  in 
the  following  formula: 

Swath  Width  =  Tan(Mirror Angle)*  Flying  Height  *  2 
=  Tan(27)  *  200  *  2 
=  204m 

Units.  Units  for  all  formulas  are: 

Flying  Height:  meters 
Flying  Speed:  meters  per  seeond 
Mirror  Angle:  degrees 
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1.0  INTRODUCTION 


This  document  serves  as  the  quality  assurance  project  plan  (QAPP)  for  the  Environmental 
Security  Technology  Certification  Program  (ESTCP)  Demonstration  Project  entitled  “High 
Density  LiDAR  and  Orthophotography  in  UXO  Wide  Area  Assessment”,  to  be  performed  at  the 
former  Kirtland  Precision  Bombing  Ranges  (PBR),  New  Mexico  (Figure  1-1).  The  QAPP  serves 
as  the  primary  guide  for  integrating  quality  assurance  (QA)  and  quality  control  (QC)  procedures 
and  functions  with  data  collection  activities  described  in  the  Performance  Plan  to  ensure  that 
technically  sound  and  defensible  data  are  collected  and  meet  project  objectives.  The  QAPP  was 
developed  in  conjunction  with  the  Demonstration  Plan,  and  health  and  safety  plan  (HASP). 

By  reference,  this  QAPP  relies  upon  the  management  policies,  objectives,  principles,  and  general 
procedures  presented  in  the  URS  Group,  Inc.  (URS)  Quality  Management  Plan  (URSG  1998). 
The  quality  management  plan  establishes  the  framework  and  general  criteria  for  implementing 
the  URS  quality  system. 

Figure  1-1  Kirtland  AFB  PBS  Vicinity  Map 
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2.0  PROJECT  DESCRIPTION 


The  WAA  study  area  for  project  UX-0534  consists  of  5,000  acres  of  the  former  Kirtland 
Precision  Bombing  Ranges  located  in  the  West  Mesa  area  of  Bernalillo  County,  New  Mexico, 
about  10  miles  west  of  Albuquerque.  The  WAA  study  area  was  selected  to  in  accordance  with 
WAA  Pilot  Program  goals  and  includes  several  known  WWII-era  targets  as  well  as  undeveloped 
land  surrounding  the  targets.  Project  UX-0534  explores  the  use  of  LiDAR  and  high-resolution 
orthophotography  as  a  means  to  locate  previously  undocumented  MRS,  delineate  the  boundaries 
of  MRS,  identify  site  constraints  on  ground-based  geophysical  surveys,  and  identify  munitions- 
related  features  such  as  craters,  targets,  burial  pits,  and  ground  and  vegetation  disturbances. 
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3.0  PROJECT  ORGANIZATION  AND  QUALITY  ASSURANCE 

RESPONSIBILITIES 


3.1  PROJECT  ORGANIZATION 

The  project-specific  organization,  shown  in  Figure  3-1,  indicates  key  positions  along  with  lines 
of  authority  and  lines  of  communication  and  coordination.  The  names  of  individuals  filling  key 
roles  on  the  project,  along  with  their  phone  numbers,  are  provided  in  Table  3-1 .  The 
responsibilities  and  authorities  of  these  key  positions  as  they  relate  to  project  QA/QC  are 
described  in  the  following  subsections. 

It  is  essential  that  all  individuals  have  defined  responsibilities  for  their  functional  areas  and  that 
they  are  clearly  aware  of  the  entire  project  organization  and  the  interrelationships  of  various 
roles.  Because  this  discussion  covers  the  project  organization,  those  senior  officials,  corporate 
managers,  and  administrators  whose  positions  are  not  functionally  involved  with  data  generation, 
data  use,  or  decision  making  are  not  included. 

QA  personnel  have  sufficient  authority,  access  to  work  areas,  and  organizational  freedom  to 
identify  quality  problems;  to  initiate,  recommend,  or  provide  solutions  to  problems  through 
established  channels;  and  to  verify  solution  implementation.  They  ensure  that  all  work, 
including  any  processing  of  information,  delivery  of  products,  and  installation  or  use  of 
equipment,  is  reviewed  in  accordance  with  the  QC  objectives  and  that  all  deficiencies  and 
nonconformances  are  corrected.  QA  personnel  have  direct  access  to  senior  management  so  that 
the  required  authority  is  available,  when  needed,  to  carry  out  QA  duties. 

The  project  planning  team  will  include  the  team  members  described  below. 

3.1.1  ESTCP  Project  Manager 

The  ESTCP-assigned  Project  Manager  (ESTCP  PM)  is  responsible  for  coordinating  all  project- 
related  activities  on  behalf  of  ESTCP.  A  major  component  of  this  position  involves  directing  the 
URS  Principle  Investigator  (PI)  in  the  execution  of  the  work  and  the  submission  of  deliverables 
as  scheduled.  Specific  responsibilities  of  the  ESTCP  PM  are  as  follows: 

•  Oversee  all  project  activities. 

•  Provide  the  scope  of  work  to  the  URS  PI. 

•  Review  and  approve  project  plans  and  coordinate  the  review  within  ESTCP  and 
other  appropriate  organizations,  as  necessary. 

•  Review  and  approve  the  QAPP  and  all  other  components  of  the  project  plan. 
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•  Ensure  that  the  QAPP  and  associated  reports  are  transmitted  to  appropriate 
ESTCP  personnel  and  organizations. 

•  Transmit  to  the  EIRS  PI  review  comments  associated  with  the  project  plan  from 
ESTCP  and  other  organizations. 

•  Ensure  that  the  URS  PI  addresses  ESTCP  and  agency  review  comments  and  takes 
appropriate  action. 

3.1,2  URS  Principle  Investigator 

The  URS  PI  is  responsible  for  providing  senior  leadership  and  expertise  to  the  URS  and 
subcontract  staff  and  for  maintaining  a  broad  perspective  of  priorities.  The  URS  PI  reports  to 
and  obtains  technical  direction  and  assistance  from  the  ESTCP  PM  and  is  responsible  for 
monitoring  and  documenting  the  quality  of  all  work  produced  by  the  project  team,  which 
includes  the  URS  staff  and  subcontractors.  The  fundamental  goal  of  this  position  is  to  produce  a 
quality  work  product  within  the  allotted  schedule  and  budget.  Duties  include  executing  all 
phases  of  the  project  and  efficiently  applying  the  full  resources  of  the  project  team  in  accordance 
with  the  project  plans.  Specific  responsibilities  of  the  URS  PI  are  as  follows: 

•  Identify  the  need  for  and  expectations  regarding  services  to  be  provided  and, 
when  necessary,  negotiate  acceptable  scopes  of  work. 

•  Provide  input  and  technical  expertise  to  the  ESTCP  PM  on  developing  or 
establishing  project  objectives,  data  quality  objectives  (DQOs),  sampling 
rationale,  and  data  analysis  and  assessment  methods. 

•  Prepare  and  implement  the  Technology  Demonstration  Plan  (which  incorporates 
applicable  QAPP  elements)  and  prepare  reports  as  required  by  ESTCP. 

•  Ensure  that  the  QAPP  and  standard  operating  procedures  (SOPs)  are  available  and 
in  use  for  activities  that  affect  product  quality  and  that  assigned  staff  have  been 
trained  in  the  implementation  of  the  QAPP  and  SOPs. 

•  Ensure  that  the  best  available  technology  is  being  applied  to  reduce  potential 
waste  and  inefficiencies  and  that  the  best  known  processes  are  in  use. 

•  Ensure  that  appropriate  procedures  for  data  collection,  data  processing,  and  data 
analysis  are  followed  and  that  correct  QC  checks  are  implemented. 

•  Perform  readiness  reviews  and  monitor  the  progress  of  work  in  relation  to  scope, 
cost,  and  schedule. 
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•  Monitor  team  and  subcontractors  for  compliance  with  project  and  data  quality 
requirements,  records,  costs,  and  progress  of  the  work;  replan  and  reschedule 
work  tasks  as  appropriate. 

•  Review  and  approve  calculations  and  data  processing  methods  to  ensure  that  data 
reduction  is  performed  in  a  manner  that  produces  quality  products. 

•  Verify  data  quality,  test  results,  equipment  calibrations,  and  QC  documentation; 
maintain  and  regularly  review  all  QC  records. 

•  Ensure  that  all  key  decisions  and  project  deliverables  are  subjected  to  independent 
technical  review  by  qualified  personnel  within  the  timeframe  of  the  project 
schedule. 

•  Report  QA  problems  to  the  Quality  Assurance  Officer  (QAO)  for  review  and/or 
resolution. 

•  Assess  completion  of  work  in  accordance  with  ESTCP. 

3,1.3  URS  Quality  Assurance  Officer 

The  URS  Quality  Assurance  Officer  (QAO)  communicates  with  the  URS  PI  regarding  the 
contract  and  additionally  has  direct  reporting  access  to  the  URS  Corporate  QA  Director  on 
quality-related  matters.  The  QAO  is  responsible  for  development,  implementation,  and 
maintenance  of  the  comprehensive  quality  system.  Responsibilities  of  this  position  include 
communicating  with  all  levels  of  program  and  project  management  to  ensure  that  a  quality 
product  is  produced  for  delivery.  Project-specific  responsibilities  of  the  QAO  or  designee  are  as 
follows: 

•  Respond  to  QA  needs,  resolve  problems,  and  answer  requests  for  guidance  or 
assistance. 

•  Provide  guidance  to  the  URS  PI  in  the  development  of  project  plans,  including  the 
QAPP. 

•  Review  and  approve  the  project  plans. 

•  Together  with  the  URS  PI,  assign  competent,  qualified  independent  reviewers  to 
review  the  technical  adequacy  of  deliverables. 

•  Track  the  progress  and  completion  of  the  review  and  approval  process. 

•  Ensure  that  ESTCP  protocols  and  procedures,  as  well  as  SOPs,  are  being 
followed. 
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•  Review  the  implementation  of  the  Teehnology  Demonstration  Plan  and  the 
adequacy  of  the  data  or  products  generated  based  on  quality  objectives. 

•  Initiate  field  and  office  audits,  if  necessary. 

•  Review  audit  and  nonconformance  reports  to  determine  areas  of  poor  quality  or 
failure  to  adhere  to  established  procedures. 

•  Confer  with  the  audited  entity  on  the  steps  to  be  taken  for  corrective  actions  and 
track  nonconformance  until  it  has  been  corrected;  evaluate  the  adequacy  and 
completeness  of  the  action  taken;  confer  with  the  URS  PI  to  resolve  an  inadequate 
corrective  action;  and  confirm  the  adequacy  and  implementation  of  the  response 
action. 

•  Upon  detection  and  identification  of  an  immediate  adverse  condition  affecting  the 
quality  of  results,  suspend  or  stop  work  with  the  concurrence  of  the  URS  PI  and 
the  ESTCP  PM. 

3.1.4  URS  Data  Evaluation  and  GIS  Lead 

The  URS  Data  Evaluation  and  GIS  Eead  is  responsible  for  managing  any  project  task  requiring 
specialized  technical  expertise,  related  to  the  collection,  assessment,  analysis  and  reporting  of 
EiDAR  and  orthophoto  data.  Specific  responsibilities  are  as  follows: 

•  Provide  technical  direction  to  URS  technical  staff  in  the  specific  area  of  EiDAR 
and  orthophoto  data  collection  and  processing. 

•  Prepare  a  statement  of  work  (SOW)  for  use  in  procurement  of  direct 
subcontracted  services  including  EiDAR  and  orthophotography. 

•  Serve  as  a  point  of  contact  for  direct  subcontractors. 

•  Relay  to  the  URS  PI  project  information  related  to  EiDAR  and  orthophoto  data 
collection  and  processing. 

•  Coordinate  data  collection  activities  to  be  consistent  with  information 
requirements. 

•  Resolve  problems  with  received  data  and/or  analysis. 

•  Oversee  evaluation  of  data  received  from  subcontractors  in  accordance  with  the 
project  requirements  and  DQOs. 
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•  Supervise  the  ereation  of  analytieal  produets  based  on  data  provided  by  direet 
subeontraetor(s). 

•  Ensure  that  data  are  eorrectly  reported  in  appropriate  data  formats. 

•  Prepare  or  oversee  the  preparation  of  portions  of  the  final  report  that  summarize 
data  results  and  present  eonelusions. 

•  Prepare  and  file  required  URS  QA/QC  review  forms,  doeumenting  QA/QC 
aetivities  related  to  data  analysis  performed  by  URS. 

3,1.5  Subcontractor 

One  direet  subeontractor,  Terra  Remote  Sensing,  Ine.  (TRSI)  will  be  used  by  URS  during  the 
demonstration.  This  subeontraetor  will  eolleet  LiDAR  and  photo  image  data  in  the  field,  and 
will  also  perform  initial  data  proeessing  to  ereate  LiDAR  points  and  orthophotos.  The 
subeontraetor  will  be  direeted  by  the  URS  PI  or  their  designee  and  apprised  of  projeet 
requirements  as  appropriate  for  their  assigned  tasks.  In  addition,  the  subeontractor  will  have  the 
following  responsibilities: 

•  Implement  the  data  collection  elements  of  the  Technology  Demonstration  Plan 
and  perform  initial  data  processing  elements  of  the  Plan. 

•  Develop  and  maintain  flight  logs,  calibration  and  accuracy  reports,  and  other  field 
activity  files,  forms,  and  logbooks  as  detailed  in  this  QAPP. 

•  Implement  technical  procedures  applicable  to  tasks,  including  calibration  and 
maintenance  of  field  equipment. 

•  Subcontract  as  necessary  for  specific  equipment  such  as  helicopter  services,  with 
the  approval  of  URS. 

•  Provide  a  health  and  safety  plan  appropriate  to  their  activities  and  to  those  of  their 
subcontractors. 

•  Report  any  problems  or  deviations  from  project  plans  to  URS  PI. 
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Table  3-1 
Project  Contacts 


Key  Role 

Name 

Telephone 

ESTCP 

ESTCP  Contracting  Officer 

Lynn  Harper 

(703)  428-6551 

ESTCP  Project  Manager  (ESTCP  PM)‘ 

Herb  Nelson 

(202)  767-3686 

On-Site  Emergencies  | 

Any  emergency 

911 

Ambulance 

911 

Kirtland  PER  /  Double  Eagle  Airport  Manager 

(505)  244-7888 

Fire 

911  or 

University  of  New  Mexico  Hospital,  Albuquerque 

(505)  272-2111 

URS  Group,  Inc.  | 

URS  Principal  Investigator 

Tom  Tomczyk 

(206)  438-2137 

URS  Data  Evaluation  and  GIS  Lead 

Dale  Bennett 

(206)  438-2026 

URS  Quality  Assurance  Officer 

George  Carlson 

(503)  478-7661 

URS  Health  and  Safety  Manager 

Mark  Litzinger 

(206)  438-2199 

TRSI.  1 

TRSI  Technical  Lead 

Dave  Neufeldt 

(250)  656-0931 

*  In  the  event  of  an  emergency,  the  URS  PI  and  the  ESTCP  PM  are  to  be  notified  immediately. 
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3.2  PROJECT  SCHEDULE 


The  project  schedule  is  provided  in  Table  3-2. 

Table  3-2, 
Project  Schedule 


Activity 

Start  Date 

End  Date 

Mobilization 

August  8,  2005 

August  8,  2005 

Field  data  collection 

August  9,  2005 

August  11,  2005 

Complete  initial  data  processing 
and  deliver  field  data  to  URS 

September  23,  2005 

September  23,  2005 

Completion  of  initial  data 
deliverables  and  transfer  to  ESTCP 

October  7,  2005 

October  7,  2005 

Data  analysis.  GIS-based 
analysis  of  LiDAR  and  image 
data  for  features;  area 
classification. 

November  30,  2005 

November  30,  2006 

Demonstration  Report  with 
Survey  Data 

February  1,  2006 

February  1,  2006 

Draft  final  report 

June  1,  2006 

June  1,  2006 

Final  report 

August  1,  2006 

August  1,  2006 
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4.0  DATA  QUALITY  PARAMETERS 


Data  Quality  Objectives  (DQOs)  are  qualitative  or  quantitative  statements  of  the  precision  (a 
measure  of  the  random  error),  bias  (a  measure  of  systematic  error),  representativeness, 
completeness,  and  comparability  necessary  for  the  data  to  serve  the  project  objectives.  During 
implementation  of  the  demonstration,  field  and  analytic  data  will  be  generated.  These  data  will 
be  evaluated  to  determine  that  DQOs  have  been  met  and  that  the  data  are  usable  for  the  project 
objectives. 


4,1  ANALYTICAL  OBJECTIVE  AND  RATIONALE 

The  objective  of  the  demonstration  is  to  accurately  and  systematically  locate  Munitions 
Response  Sites  (MRS)  and  ground  features  associated  with  the  use  of  munitions  and  explosives 
of  concern  (MEC).  A  secondary  objective  is  to  develop  realistic  cost/benefit  data  regarding  the 
appropriate  densities  of  LiDAR  and  orthophotographs  in  the  detection  and  characterization  of 
MRS  and  MEC-related  ground  features.  The  primary  means  to  achieve  these  objectives  is 
through  the  development  and  analysis  of  detailed  models  of  the  ground  surface  based  on  several 
different  densities  of  LiDAR  points,  and  of  orthophoto  images  at  different  pixel  sizes. 
Consequently,  analytical  objectives  are  designed  to  ensure  that  LiDAR  and  orthophotography 
products  created  accurately  reflect  the  capabilities  of  each  technology,  free  of  bias  and  error 
which  will  distort  the  experimental  results. 

Eurther,  the  products  of  the  LiDAR  and  orthophotography  demonstration  will  be  used  in 
subsequent  phases  of  the  overall  ESTCP  UXO  pilot  project.  Consequently,  a  further  analytic 
objective  is  to  assure  that  the  spatial  coordinates  of  the  LiDAR  and  orthophoto  data  are 
controlled,  to  the  extent  allowed  by  each  technology,  to  on-site  survey  control  points  that  will  be 
used  by  other  demonstrators. 


4,2  QUALITY  ASSURANCE  MEASUREMENTS 

The  quality  of  the  field  data  will  be  evaluated  based  on  successful  calibration  of  each  instrument 
supplying  the  data  based  on  the  manufacturer’s  stated  accuracy  and  precision  of  the  instrument. 
The  quality  of  data  processing  and  data  analysis  steps  will  be  evaluated  based  on  the  precision, 
bias,  representativeness,  completeness,  and  comparability  of  the  data  generated  by  each  type  of 
analysis.  These  data  assessment  parameters  are  described  in  the  following  subsections. 

4,2.1  Precision 

Precision  is  a  measure  of  the  scatter  in  the  data  due  to  random  error.  Lor  airborne  remote  sensing 
technologies,  the  major  sources  of  random  error  are  the  sampling  rate  of  the  sensors  in  relation  to 
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the  positional  data  provided  by  the  Inertial  Measurement  Unit  (IMU)  and  Global  Positioning 
System  (GPS).  Sampling  and  analytieal  preeision  is  expressed  as  the  relative  differenee  in 
position  between  the  LiDAR  and  orthophoto  data  and  ground-based  survey  control  points, 
expressed  in  centimeters. 

4.2.2  Bias 

Bias  is  a  measure  of  the  difference  between  the  analytical  results  for  a  parameter  and  the  true 
value  due  to  systematic  errors.  The  primary  source  of  systematic  error  is  equipment  calibration. 
Secondarily,  systematic  error  can  be  created  by  improper  flight  line  layout,  which  can  result  in 
uneven  data  collection  in  some  types  of  terrain.  Since  the  Kirtland  PBR  demonstration  site  is 
relatively  flat,  flight  line  layout  is  not  expected  to  be  a  problem.  Bias  is  expressed  as  positional 
differences  in  LiDAR  data  between  one  flight  line  and  the  next,  or  between  LiDAR  and 
orthophoto  positions,  that  are  beyond  stated  standards  and  cannot  be  adjusted  out. 

4.2.3  Representativeness 

Representativeness  is  a  qualitative  parameter  that  expresses  the  degree  to  which  sample  data 
accurately  and  precisely  represents  a  characteristic  of  a  population,  parameter  variations  at  a 
sampling  point,  or  an  environmental  condition.  Representativeness  of  the  environmental 
conditions  at  the  time  of  sampling  is  achieved  by  selecting  sampling  locations,  methods,  and  data 
collection  times  so  that  the  data  describe  the  site  conditions  that  the  project  seeks  to  evaluate.  In 
the  context  of  LiDAR  and  orthophotography,  representative  sampling  is  influenced  by  proper 
choice  of  seasonal  vegetative  conditions  (for  more  vegetated  sites),  and  proper  choice  of  sun  angle 
to  best  reveal  the  features  being  sought.  During  data  processing,  representative  LiDAR  data  is 
obtained  by  properly  separating  laser  pulses  that  reflect  from  the  ground  surface  from  those  that 
reflect  from  vegetation  or  buildings.  Representative  orthophotography  data  are  obtained  by 
performing  appropriate  color  balancing  on  orthophoto  mosaics  to  eliminate  the  effect  of  glare  or 
different  sun  angles  on  images  from  flight  lines  in  opposing  directions,  while  still  accurately 
representing  the  colors  of  the  features  being  recorded. 

4.2.4  Completeness 

Completeness  is  a  measure  of  the  number  of  valid  measurements  obtained  in  relation  to  the  total 
number  of  measurements  planned.  In  the  context  of  LiDAR  and  orthophotography, 
completeness  is  expressed  in  percentage  of  the  target  area  for  which  data  is  successfully 
collected,  and  for  which  the  data  collected  meets  project  specifications.  For  this  demonstration, 
100%  coverage  of  the  demonstration  site  is  expected. 

4.2.5  Comparability 

Comparability  is  a  qualitative  parameter  expressing  the  confidence  with  which  one  data  set  can 
be  compared  to  another.  During  data  collection  the  comparability  goal  is  achieved  by 
maintaining  consistency  in  sampling  conditions  such  as  flight  altitude,  and  equipment  settings. 
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These  are  aimed  at  having  data  from  eaeh  flight  line  and  eaeh  flight  mission  be  eomparable  to 
the  next,  and  at  having  the  LiDAR  and  orthophoto  data  positions  be  well  eontrolled  to  ground 
survey  eontrol  points  to  be  used  by  other  demonstrators. 

During  data  proeessing  and  analysis,  eomparability  is  maintained  through  establishment  and 
execution  of  appropriate  data  management  and  data  format  standards.  This  results  in  data  that  is 
organized  in  a  consistent  manner,  with  appropriate  metadata  to  enable  other  users  to  understand 
the  data  processing  steps  used  and  their  rationale,  and  in  formats  that  have  been  agreed  to  by 
other  demonstrators  and  ESTCP. 

The  following  parameters  in  Table  4-1  will  be  subject  to  QA/QC  review: 
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Table  4-1 

Data  Quality  Parameters 


Analytical 

Objective 

Metric 

Action  to  Achieve  Metric 

Sampling 
Frequency  or 
Timing 

Desired  Result 

Pre-flight  Activities 

Study  area 

boundary 

delineation 

Site  Boundary  polygon 
characteristics  agreed 
upon  and  documented  to 
allow  comparison  with 
data  collected. 

Achieve  stakeholder 
agreement  to  boundary 
parameters. 

Once,  start  of 
program. 

Document  site 
boundary  for 
measurement  of 
future  boundary 
metrics. 

Survey  control 
point 

confirmation 

measurement 

Confirm  coordinates  of 
survey  control  points 
within  at  least  3^“*  order 
accuracy. 

Perform  and  record  GPS 
survey  (static  or  kinematic). 

Pre-flight  (or 
during  on-site 
acquisition). 

LiDAR  Data  Collection  and  Processing 

Sensor 

calibration 

Resolve 

roll/pitch/heading  for 
installation. 

Perform  opposing  direction 
and  orthogonal  passes  over 
baseline.  Compare  with 
nominal  values  from  standard 
installation. 

Prior  to  each 
flight  day. 

+!-  0.02  degrees 

Sensor  speed* 

Laser  pulse  rate  between 
50-100  kHz. 

Set  laser  pulse  speed  and  the 
altitude  of  the  low  LiDAR 
passes  depending  on  site 
conditions  to  achieve  highest 
possible  point  density. 

Prior  to  each 
flight  day. 

Achieve  target 
sensor  speed. 

Flight  altitude 

Flight  altitudes  of  900, 
450,  and  200  to  300  m. 

The  two  higher  altitudes 
are  designed  to  achieve 
10-cm  and  20-cm  pixel 
sizes;  however,  LiDAR 
will  still  be  collected. 

The  lowest  flight 
altitudes  will  be  adjusted 
to  achieve  the  highest 
practical  LiDAR  point 
densities. 

Establish  and  fly  appropriate 
flight  altitudes  for  the  desired 
LiDAR  point  densities  and 
orthophoto  pixel  sizes.  Lay 
out  a  series  of  flight  lines  for 
high-density  LiDAR 
collection  to  be  able  to 
respond  to  site  conditions. 

Each  flight 
line. 

+!-  50  m  from 
planned  flight 
altitudes^. 

Area  coverage 

100%  coverage  for  each 
flight. 

Establish  and  fly  flight  lines  so 
as  to  cover  the  entire  target 
area.  Data  from  each  day’s 
flights  will  be  examined  and 
data  gaps  will  be  filled. 

Each  flight 

100%  coverage. 

15%  flightline 
overlap,  50m  over 
area  boundaries. 

Appropriate  sensor  speed  is  a  site-specific  determination  based  on  site  reflectivity.  Higher  speeds 
result  in  lower  energy  per  laser  pulse,  and  thus  require  more  a  reflective  ground  surface.  Both 
ground  material  and  flight  altitude  influence  reflectivity. 

^  Flight  altitudes  significantly  above  the  performance  objective  will  fail  the  point  density  test  and 
photo  pixel  size  tests  given  below  and  will  be  repeated. 
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Table  4-1  (Continued) 
Data  Quality  Parameters 


Analytical 

Objective 

Metric 

Action  to  Achieve  Metric 

Sampling 
Frequency  or 
Timing 

Desired  Result 

Data  collection 
rate 

Accomplish  2  flights  per 
day,  each  covering  the 
full  site.  Reserve  1 
additional  day  for 

QA/QC  and  re-flights 

Establish  and  review  flight 
lines  and  flight  schedule  prior 
to  data  collection. 

N/A 

Full  data  collection 
within  planned 
schedule. 

LiDAR  point 
density 

Achieve  overall  densities 
of: 

200  m  flights  (2)  -  8 
pts/m^  each 

475  m  flight  (1)  -  3  pts/ 
m^ 

950  m  flight  (1)  - 
L5pts/m^' 

Plan  and  accomplish 
appropriate  sensor  speed, 
flight  altitude,  and  air  speed. 
Flights  more  than  10%  below 
target  point  densities  will  be 
repeated. 

Each  flight. 

Data  collection 
within  10%  of  target 
densities. 

LiDAR  flight 
line  alignment 

The  two  200  m  flights 
will  be  orthogonal. 

Appropriate  flight  lines  will  be 
designed  and  flown.  Planned 
flight  lines  will  be  submitted 
in  advance. 

Each  200  m 
altitude  flights 

Flight  lines  within 

10“  of  orthogonal. 

LiDAR  vertical 
accuracy 

Vertical  accuracy  of  +!- 
15  cm  compared  to 
ground  survey. 

Steps: 

1 .  Perform  sensor  calibration 
as  described  above 

2.  Obtain  ground  elevations  on 
identifiable  points  using 
ground  based  GPS  methods 
(static  and/or  kinematic) 

3.  Compare  ground-based  and 
airborne  elevations. 

Each  flight 

Achieve  best 
possible  match 
between  LiDAR  and 
ground-based 
elevations. 

LiDAR 

horizontal 

accuracy 

Horizontal  accuracy  of 
+!-  65  cm  compared  to 
ground  survey. 

Steps: 

1 .  Perform  sensor  calibration 
as  described  above 

2.  Obtain  ground  positions  on 
identifiable  points  using 
ground  based  GPS  methods 
(static  and/or  kinematic) 

3.  Compare  ground-based  and 
airborne  positions. 

Each  flight 

Achieve  best 
possible  match 
between  LiDAR  and 
ground-based 
positions. 

LiDAR  data 
integration  - 
flight  lines 

Achieve  flight  line  to 
flight  line  edge  match  of 
+!-  12cm. 

Review  statistics  from  LiDAR 
processing  software. 

Line  to  line 

Achieve  best 
possible  match 
between  individual 
LiDAR  flight  lines. 
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Table  4-1  (Continued) 
Data  Quality  Parameters 


Analytical 

Objective 

Metric 

Action  to  Achieve  Metric 

Sampling 
Frequency  or 
Timing 

Desired  Result 

LiDAR  point 
separation 

Remove  100%  of  large 
features,  (trees, 
buildings,  vehicles) 
Remove  small  features 
(grass,  low  brush)  to  the 
level  where  remaining 
data  cannot  distinguish 
ground  from  non-ground 
features. 

Operators  remove  non-ground 
laser  returns  through 
automated  separation  routines 
followed  by  hand  cleaning  and 
inspection. 

LiDAR  data 
set  for  each 
flight 

Satisfactory  visual 
inspection  of  surface 
model  of  the  ground 
surface. 

Orthophoto  Data  Collection  and  Processing 

Orthophoto  area 
coverage 

100%  coverage  for  each 
flight. 

Wireframes  of  “raw”  images 
are  compared  to  the  project 
boundary  to  check  for  gaps  or 
holes. 

Each  flight  day 
as  part  of 
QA/QC 
checks. 

100%  coverage  with 
sufficient  image 
overlap  for  ortho¬ 
rectification. 

Orthophoto 
flight  altitude  / 
pixel  size 

475  m  (for  10  cm  pixel 
flight) 

950  m  (for  20  cm  pixel 
flight). 

Orthophoto  pixel  size  is 
directly  related  to  flight 
altitude.  Flight  lines  are 
designed  for  the  desired  pixel 
sizes.  Flight  data  will  be 
examined  during  and  after 
each  flight  and  flight  lines 
outside  of  the  range  will  be 
repeated. 

Each  flight 

+/-  50m  of  design 
flying  altitude. 

Orthophoto 

image 

mosaicing 

No  obvious  seams 
between  images  in  the 
final  orthophoto. 

Creation  of  an  image  mosaic 
from  individual  small  images 
is  largely  an  operator 
controlled  rather  than  an 
automated  process. 

Each 

orthophoto 
composite 
image  (10  cm 
and  20  cm) 

Line  features  are 
continuous  with  no 
visible  discontinuity 
at  mosaic  seams. 

Orthophoto 
image  color 
balancing 

No  obvious  color 
imbalances  within  data 
for  each  session.^"* 

Color  balancing  is  an  operator 
controlled  process  based  on 
viewing  the  mosaic  to  identify 
any  areas  of  tonal  imbalance. 

Each 

orthophoto 
composite 
image  (10  cm 
and  20  cm) 

Continuity  of  tone 
such  that  individual 
images  are  not 
visible  in  mosaic. 

^  The  final  project  area  outline  will  determine  the  number  of  flight  lines  and  turns  necessary  to  cover  the  project 
area.  This  in  turn  will  determine  the  duration  of  each  acquisition  period.  In  order  to  ensure  relatively  constant 
sun  angle,  if  necessary,  each  image  acquisition  session  will  be  split  into  two  -  one  early  in  the  day  and  one  late 
in  the  day  in  which  the  sun-angle  is  the  same  with  the  exception  of  being  in  the  opposite  direction.  Tonal 
balance  between  the  early  and  late  sessions  may  not  be  the  same  at  the  junction  between  the  two  collection 
areas. 
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Table  4-1  (Continued) 
Data  Quality  Parameters 


Analytical 

Objective 

Metric 

Action  to  Achieve  Metric 

Sampling 
Frequency  or 
Timing 

Desired  Result 

Orthophoto 
horizontal 
alignment  to 
LiDAR 

LiDAR  and  orthophotos 
aligned  so  that  target 
features  are  not 
displaced  in  the  two  data 
sets. 

Orthorectification  is 
performed  using  the  LiDAR 
data  and  fiducial  locations  are 
control  data  sources,  followed 
by  operator  adjustment. 

Each 

orthophoto 

composite 

image. 

Orthophotos  aligned 
to  +/-  2  pixel 
widths. 

Orthophoto 
horizontal 
alignment  to 
fiducials 

Orthophotos  aligned  to 
survey  control  points  so 
that  target  features  are 
not  displaced. 

Orthorectification  is 
performed  using  the  LiDAR 
data  and  fiducial  locations  are 
control  data  sources,  followed 
by  operator  adjustment. 

Each 

orthophoto 

composite 

image. 

Orthophotos  aligned 
to  +/-  3  pixel 
widths. 

MRS  Identification  and  Analysis 

MRS 

identification'* 

Correctly  identify  all 
previously  identified 

MRS. 

Identify  and  document  MRS 
from  LiDAR  and  orthophoto 
data  sets. 

Each  LiDAR 
and  orthophoto 
data  set  and 
combinations. 

All  MRS 

MRS  false 
alarm  rate 

No  areas  incorrectly 
identified  as  MRS. 

Identify  and  document  MRS 
from  LiDAR  and  orthophoto 
data  sets. 

Each  LiDAR 
and  orthophoto 
data  set  and 
combinations. 

All  MRS 

MBS  boundary 
delineation 

Correctly  locate  MRS 
boundaries  to  +/-  15% 
of  ground-tmthed  area. 

Identify  and  document  MRS 
boundaries  from  LiDAR  and 
orthophoto  data  sets  for  a 
selected  set  of  test  MRS. 

Each  LiDAR 
and  orthophoto 
data  set  and 
combinations. 

All  MRS 

MRS  feature 
identification 

Identify  features 
presenting  as  human- 
made  (anthropogenic) 
not  including  craters 
(e.g.,  walls,  berms,  pits, 
small  buildings). 

LiDAR  and  photo  data  sets 
will  be  examined  for  linear 
features. 

Each  LiDAR 
and  orthophoto 
data  set  and 
combinations. 

90%  of  features 
identified  from 
selected  field- 
identified  features. 

MRS  feature 
identification 

Identify  craters.  Count 
90%  of  craters  over  1 .5 
m  diameter  and  .3  m 
depth. 

Automated  algorithms  will  be 
used  to  identify  and  count 
craters  using  LiDAR  data. 

Each  LiDAR 
and  orthophoto 
data  set  and 
combinations. 

90%  of  craters 
identified  outside  of 
crater  fields.  95% 
of  craters  identified 
inside  crater  fields. 

The  analysis  described  will  be  performed  for  each  separate  LiDAR  data  set  and  for  the  combined  LiDAR  data 
set,  and  for  each  orthophoto  data  set  and  for  the  combination  of  LiDAR  and  orthophotos.  The  “Desired  Resuh” 
metric  applies  to  the  combination  of  all 
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Table  4-1  (Continued) 
Data  Quality  Parameters 


Analytical 

Objective 

Metric 

Action  to  Achieve  Metric 

Sampling 
Freqnency  or 
Timing 

Desired  Result 

MRS  feature 
identification 

Identify  vegetation 
patterns  indicating 
previous  disturbance. 

Automated  algorithms  will  be 
used  to  map  vegetation  heights 
and  patterns  from  LiDAR 
data.  Results  will  be 
examined  for  linearity  or  other 
regular  shapes.  Orthophotos 
will  be  examined  for  regular 
vegetation  patterns. 

Each  LiDAR 
and  orthophoto 
data  set  and 
combinations. 

[COMING] 

MRS  feature 
identification 

Identify  established 
roads 

LiDAR  and  photo  data  sets 
will  be  examined  for  linear 
features. 

Each  LiDAR 
and  orthophoto 
data  set  and 
combinations. 

100%  of  established 
roads. 

MRS  feature 
identification 

Identify  vehicle  tracks 

LiDAR  and  photo  data  sets 
will  be  examined  for  linear 
features. 

Each  LiDAR 
and  orthophoto 
data  set  and 
combinations. 

90%  of  field- 
identified  vehicle 
tracks. 

MRS  feature 
identification 

Identify  topography  that 
can  limit  access 

LiDAR  data  sets  will  be  used 
to  map  areas  above  designated 
slope. 

Each  LiDAR 
data  set  and 
combinations. 

100%  of  areas. 

Data  management 

Data 

management 

Data  backup  and  storage 
to  achieve  redundancy 
and  security. 

Data  will  be  backed  up  to 
separate  redundant  hard  drives 
or  tape  drives. 

Daily  backup 
during  field 
and  data 
processing 
operations. 

All  data  products. 

Data  transfer 

Data  transfer  will  be  in 
appropriate  formats  and 
file  sizes  for  ESTCP  and 
Kirtland  AFB  ongoing 
use. 

Data  will  meet  US 

Government  Spatial  Data 
Standard  (SDS)  and  fully 
comply  with  Versar  EDD 
specifications. 

Each  data 
transfer. 

All  data  products. 

Data  collection 
report 

Standard  flight  reporting 
includes:  calibration  log, 
flight  log,  QA/QC  log, 
and  site  photos. 

The  data  collection  report  is  a 
standard  QA/QC  product. 

Calibration 
report:  each 
flight  day. 

Flight  log: 
each  flight. 
QA/QC  log: 
each  flight. 

Site  photos: 
whole  project. 

Full  reporting  is  a 
required  part  of 
contract 
performance. 

Data  processing 
report 

Standard  data  processing 
report  includes: 

GPS  control  ties, 
accuracy  verification 
report,  and  QA/QC 
report. 

The  data  processing  report  is  a 
standard  QA/QC  product. 

Each  LiDAR 
and  orthophoto 
data  set. 

Full  reporting  is  a 
required  part  of 
contract 
performance. 
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Table  4-1  (Continued) 
Data  Quality  Parameters 


Analytical 

Objective 

Metric 

Action  to  Achieve  Metric 

Sampling 
Freqnency  or 
Timing 

Desired  Result 

Metadata 

Metadata  to  aeeurately 
deseribe  data  format  and 
proeessing  steps. 

Data  will  meet  US 

Government  Spatial  Data 
Standard  (SDS)  and  fully 
eomply  with  Versar  EDD 
speeifieations  ineluding 
metadata  standards. 

Eaeh  data 
transfer. 

All  data  produets. 

QA/QC 

All  data  and  derived 
produets  will  be  subjeet 
to  appropriate  QA/QC 
review. 

Data  proeessing  will  follow 
the  QA/QC  plan  deseribed 
herein. 

Eaeh  data 
transfer. 

All  data  produets. 

Data  delivery 

All  data  will  be  delivered 
in  a  timely  and  easy-to- 
transfer  manner. 

Data  deliverables  will  be  made 
using  ftp  where  possible,  but 
in  all  eases  will  be  followed 
up  with  delivery  on  physieal 
media,  primarily  external  hard 
drives. 

Eaeh  data 
transfer. 

All  data  produets. 
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5.0  CALIBRATION  PROCEDURES,  QUALITY  CONTROL  CHECKS,  AND 

CORRECTIVE  ACTION 


Upon  arrival  at  the  site  a  suitable  GPS  base  station  will  be  established  either  at  one  of  the  survey 
eontrol  points  or  near  the  hanger.  If  a  location  near  the  hanger  is  chosen  due  to  security  or 
operational  reasons,  a  GPS  control  survey  will  be  conducted  to  tie  the  base  station  into  the 
control  points.  Targets  will  also  be  placed  on  the  survey  control  points  to  facilitate  identification 
in  the  digital  images  and  the  intensity  image  of  the  LiDAR  data.  Positions  of  the  survey  control 
points  will  be  verified  and  any  discrepancies  immediately  noted  and  transmitted  to  the  URS  PI 
and  ESTCP  PM. 

Initial  equipment  calibration  will  be  performed  following  equipment  installation  to  the 
helicopter.  Calibration  will  be  done  by  means  of  a  calibration  flight,  during  which  test  data  will 
be  collected  to  determine  optimal  system  operation  parameters  for  data  acquisition.  This  will 
involve  collecting  data  at  varying  heights  above  ground  between  200m  and  300m  with  variations 
in  the  laser  pulse  rate  (PRF). 

Equipment  calibration  will  be  checked  prior  to  each  day’s  flight  missions.  GPS  data  will  be 
processed  and  checked  each  day  for  DOPS  and  quality  of  solution.  The  redundant  base  station 
will  be  processed  to  verify  the  solution  and  obtain  the  best-fit  positions. 

The  following  QA/QC  checks  will  be  made  to  verify  equipment  calibration  and  data  quality: 

a)  EiDAR 

Following  each  day’s  flights,  the  initial  calibration  values  will  be  used  to  process  the  airborne 
data  to  obtain  the  EiDAR  points.  The  points  will  be  viewed  visually  to  inspect  the  coverage  with 
respect  to  the  overall  project  boundary  in  order  to  verify  complete  coverage.  Flight  lines  can  be 
displayed  in  different  colors  to  allow  measurement  of  overlap  as  well  as  checking  of  height 
variations  in  the  overlap  region.  Point  densities  will  be  calculated  on  sample  areas  throughout 
the  project  to  verify  compliance  with  specifications.  Visualization  tools  are  used  to  manipulate 
the  points  to  check  for  comparison  of  features  and  general  alignment  of  the  data. 

b)  Imagery 

Following  each  day’s  flights,  “raw”  images  will  be  checked  individually  for  completeness  and 
exposure.  Images  with  significantly  abnormal  exposure  such  as  bright  spots  due  sun  glint  off 
ground  objects  will  be  flagged  for  re-flight.  Using  an  initial  ground  model  from  the  EiDAR  data, 
the  wire  frame  outlines  of  the  individual  images  will  be  draped  onto  the  ground  model  to 
determine  the  true  coverage  of  each  photo  image.  The  image  coverage  will  be  reviewed  for  gaps 
within  the  flight  lines  and  extents  to  the  project  perimeter.  Measurements  are  also  made  on  the 
frames  to  validate  compliance  of  end-lap  and  side-lap  with  project  specifications. 

Following  return  of  the  data  to  the  office,  field  calibration  will  be  reviewed  and  fine-tuned. 
Following  QC  checks,  calibration  factors  will  be  applied  to  laser  range,  GPS,  and  INS  data  to 
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produce  x,y,z  values  for  eaeh  LiDAR  point.  Points  are  then  transformed  into  the  delivery  datum 
and  map  projeetion.  Subsequently,  the  points  are  separated  into  returns  from  ground,  vegetation 
and  buildings.  Following  point  separation,  separate  x,y,z  files  (in  ASCII  eomma-delimited 
format)  will  be  ereated  for  ground,  vegetation  and  building  returns  for  eaeh  of  the  four  flights 
and  delivered  to  URS.  Calibration  of  orthophoto  images  to  the  LiDAR  surfaee  model  and  the 
survey  eontrol  points  will  be  performed  using  standard  orthoreetifieation  software  methods. 
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6.0  DEMONSTRATION  PROCEDURES 


6.1  FIELD  ANALYTICAL  PROCEDURES 

Technology  startup  includes  setting  out  of  GPS  receivers  over  base  stations  and  starting.  The 
helicopter  system  is  started  including  sub-systems  of  GPS,  INS,  navigation  control,  and  logging 
systems.  Checks  are  made  to  ensure  correct  operation,  flight  line  data  is  uploaded  and  verified. 
Once  ground  operation  is  verified  the  helicopter  commences  flight  to  the  lines  to  be  flown, 
during  this  time  while  aloft,  the  LiDAR  system  operation  is  verified  (this  is  done  aloft  to  prevent 
damage  to  the  system  firing  too  close  to  the  ground).  Following  verification  of  system  operation, 
data  collection  is  then  undertaken. 

During  data  collection  all  systems  are  monitored  to  verify  continuous  correct  operation.  Should 
a  failure  occur,  airborne  procedures  to  rectify  the  system  will  be  implemented  including  resetting 
various  portions  of  the  system.  Should  these  procedures  not  be  effective,  depending  on  the 
nature  of  the  failure,  the  helicopter  will  land  in  the  project  area  or  return  to  the  base  for  further 
corrective  actions. 


6.2  LABORATORY  ANALYTICAL  PROCEDURES 

Laboratory  procedures  include  a  further  review  of  all  calibration  data  to  refine  the  calibration  for 
the  system  installation.  Following  this  checks  are  made  on  the  fiducials  and  any  other  available 
control  points  or  ground  data.  If  the  checks  meet  specifications,  the  LiDAR  data  is  passed  on  to 
the  TM  (Terrain  Mapping)  group  for  processing  to  extract  the  bare  earth  model  while  the  image 
data  is  passed  to  the  Ortho  group  for  creation  of  the  mosaic’s. 

Creation  of  the  hare-earth  model  through  classification  of  the  raw  LiDAR  points  includes  both 
automated  process  and  manual  reviews  of  the  data.  Surfaces  and  contours  may  be  created  to 
identify  areas  that  have  not  been  correctly  classified. 
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7.0  CALCULATION  OF  DATA  QUALITY  INDICATORS 


This  section  outlines  the  procedures  for  verifying  the  accuracy  of  the  data  reduction  and  analysis 
process  and  the  methods  used  to  ensure  that  data  transfer  is  error  free  (or  has  an  admissible  error 
rate),  that  no  information  is  lost  in  the  transfer  process,  and  that  the  output  is  completely 
recoverable  from  the  input.  To  reduce  the  risks  associated  with  data  transfer,  this  process  is  kept 
to  a  minimum.  Data  are  reduced  either  manually  on  calculation  sheets  or  by  computer  on 
formatted  printouts.  The  following  responsibilities  will  be  delegated  in  the  data  reduction 
process: 

•  Technical  personnel  will  document  and  review  their  own  work  and  are 
accountable  for  its  correctness. 

•  Major  calculations  will  receive  both  a  method  and  an  arithmetic  check  by  an 
independent  checker.  The  checker  is  accountable  for  the  correctness  of  the 
checking  process.  The  checker  is  also  responsible  for  becoming  familiar  with  the 
project  requirements  and  criteria;  and  bringing  to  the  URS  Pi’s  attention  any 
problems  uncovered. 

•  The  URS  PI  is  responsible  for  ensuring  that  data  reduction  is  performed  in  a 
manner  that  produces  quality  data  through  review  and  approval  of  calculations. 

In  general,  any  personnel  performing  review  or  oversight  shall  provide  resumes  or  equivalent 
documentation  demonstrating  their  qualifications  and  experience  needed  to  perform  their  duties. 
In  addition,  each  assessor  shall  be  independent  of  the  process  under  evaluation.  Independent 
Technical  Review  for  this  project  will  be  performed  by  ESTCP  or  its  designated  subcontractors. 
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7.1  DATA  QUALITY  CALCULATIONS 

The  following  methods  will  be  used  to  calculate  data  quality. 

Table  7-1 

Data  Quality  Metric  Calculation 


Analytical 

Objective 

Metric 

Action  to  Achieve  Metric 

Desired 

Result 

Calculation  Method 

Survey 
control  point 
confirmation 
measurement 

Confirm  coordinates 
of  survey  control 
points  within  at  least 
3"^  order  accuracy. 

Perform  and  record  GPS  survey 
(static  or  kinematic). 

3^**  order 
accuracy 

(x,y,z  locations  of 
confirmation  survey)  - 
(reported  x,y,z 
locations) 

Sensor 

calibration 

Resolve 

roll/pitch/heading 
for  installation. 

Perform  opposing  direction  and 
orthogonal  passes  over  baseline. 
Compare  with  nominal  values 
from  standard  installation. 

+/-  0.02 
degrees 

(x,y,z  locations  of 
control  points)  -  (x,y,z 
locations  from  sensors) 

Sensor  speed 

Laser  pulse  rate 
between  50-100 
kFlz. 

Set  laser  pulse  speed  and  the 
altitude  of  the  low  LiDAR 
passes  depending  on  site 
conditions  to  achieve  highest 
possible  point  density. 

Achieve  target 
sensor  speed. 

Examination  of  sensor 
output,  calculation  of 
point  densities 

Flight  altitude 

Flight  altitudes  of 

900,  450,  and  200  to 
300  m. 

Establish  and  fly  appropriate 
flight  altitudes  for  the  desired 
LiDAR  point  densities  and 
orthophoto  pixel  sizes.  Lay  out 
a  series  of  flight  lines  for  high- 
density  LiDAR  collection  to  be 
able  to  respond  to  site 
conditions. 

+/-  50  m  from 
planned  flight 
altitudes. 

In-flight  monitoring, 
examination  of  GPS 
flight  line  data. 

Area 

coverage 

100%  coverage  for 
each  flight. 

Establish  and  fly  flight  lines  so 
as  to  cover  the  entire  target  area. 
Data  from  each  day’s  flights  will 
be  examined  and  “holes”  filled. 

100% 

coverage. 

15%  fiighthne 
overlap,  15% 
fiighthne 
length  over 
area 

boundaries. 

(Area  flown)  -  (Area  of 
demonstration  site)  plus 
visual  inspection 

LiDAR  point 
density 

Achieve  overall 
densities  of: 

200-  to  300-m 
flights  (2)  -  8  pts/m^ 
each 

475-m  flight  (1)  -  3 
pts/  m^ 

950-m  flight  (1)  -  2 
pts/ 

Plan  and  accomplish  appropriate 
sensor  speed,  flight  altitude,  and 
air  speed.  Flights  more  than 

10%  below  target  point  densities 
will  be  repeated. 

Data  collection 
within  10%  of 
target 
densities. 

Density  =  (#  of 
points)/(area  in  meters) 

LiDAR  flight 
line  alignment 

The  two  200  m 
flights  will  be 
orthogonal. 

Appropriate  flight  lines  will  be 
designed  and  flown.  Planned 
flight  lines  will  be  submitted  in 
advance. 

Flight  lines 
within  10°  of 
orthogonal. 

Angle  of  intersection 
will  be  measured. 
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Table  7-1)  Continued) 

Data  Quality  Metric  Calculation 


Analytical 

Objective 

Metric 

Action  to  Achieve  Metric 

Desired 

Result 

Calculation  Method 

LiDAR 

vertical 

accuracy 

Vertical  accuracy  of 
+/-  15  cm  compared 
to  ground  survey. 

Steps: 

1 .  Perform  sensor  calibration  as 
described  above 

2.  Obtain  ground  elevations  on 
identifiable  points  using  ground 
based  GPS  methods  (static 
and/or  kinematic) 

3.  Compare  ground-based  and 
airborne  elevations. 

Achieve  best 
possible  match 
between 

LiDAR  and 

ground-based 

elevations. 

(z  locations  of  control 
points)  -  (z  locations 
from  sensors) 

LiDAR 

horizontal 

accuracy 

Horizontal  accuracy 
of  +/-  65  cm 
compared  to  ground 
survey. 

Steps: 

1 .  Perform  sensor  calibration  as 
described  above 

2.  Obtain  ground  elevations  on 
identifiable  points  using  ground 
based  GPS  methods  (static 
and/or  kinematic) 

3.  Compare  ground-based  and 
airborne  elevations. 

Achieve  best 
possible  match 
between 

LiDAR  and 

ground-based 

elevations. 

(x,y  locations  of  control 
points)  -  (x,y  locations 
from  sensors) 

LiDAR  data 
integration  - 
flight  lines 

Achieve  flight  line 
to  flight  line  edge 
match  of  +/-  12cm. 

Review  statistics  from  LiDAR 
processing  software. 

Achieve  best 
possible  match 
between 
individual 
LiDAR  flight 
lines. 

(x,y,z  locations  from 
flight  lines  compared 
visually  and  through 
automated  software 
matching 

LiDAR  point 
separation 

Remove  100%  of 
large  features, 

(trees,  buildings, 
vehicles) 

Remove  small 
features  (grass,  low 
brush)  to  the  level 
where  remaining 
data  cannot 
distinguish  ground 
from  non-ground 
features. 

Operators  remove  non-ground 
laser  returns  through  automated 
separation  routines  followed  by 
hand  cleaning  and  inspection. 

Satisfactory 

visual 

inspection  of 
surface  model 
of  the  ground 
surface. 

Visual  inspection 
following  automated 
and  operator 
implemented  separation 

Orthophoto 
area  coverage 

100%  coverage  for 
each  flight. 

Wireframes  of  “raw”  images  are 
compared  to  the  project 
boundary  to  check  for  gaps  or 
holes. 

100% 

coverage  with 
sufficient 
image  overlap 
for  ortho - 
rectification. 

(Area  flown)  -  (Area  of 
demonstration  site)  plus 
visual  inspection 
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Table  7-1)  Continued) 

Data  Quality  Metric  Calculation 


Analytical 

Objective 

Metric 

Action  to  Achieve  Metric 

Desired 

Result 

Calculation  Method 

Orthophoto 
flight  altitude 
/  pixel  size 

475  m  (for  10  cm 
pixel  flight) 

950  m  (for  20  cm 
pixel  flight). 

Orthophoto  pixel  size  is  directly 
related  to  flight  altitude.  Flight 
lines  are  designed  for  the  desired 
pixel  sizes.  Flight  data  will  be 
examined  during  and  after  each 
flight  and  flight  lines  outside  of 
the  range  will  be  repeated. 

+/-  50m  of 
design  flying 
altitude. 

In-flight  monitoring, 
examination  of  GPS 
flight  line  data. 

Orthophoto 

image 

mosaicing 

No  obvious  seams 
between  images  in 
the  final  orthophoto. 

Creation  of  an  image  mosaic 
from  individual  small  images  is 
largely  an  operator  controlled 
rather  than  an  automated 
process. 

Line  features 
are  continuous 
with  no  visible 
discontinuity 
at  mosaic 

seams. 

Visual  inspection 

Orthophoto 
image  color 
balancing 

No  obvious  color 
imbalances. 

Color  balancing  is  an  operator 
controlled  process  based  on 
viewing  the  mosaic  to  identify 
any  areas  of  tonal  imbalance. 

Continuity  of 
tone  such  that 
individual 
images  are  not 
visible  in 
mosaic. 

Visual  inspection. 

Orthophoto 
horizontal 
alignment  to 
LiDAR 

LiDAR  and 
orthophotos  aligned 
so  that  target 
features  are  not 
displaced  in  the  two 
data  sets. 

Orthorectification  is  performed 
using  the  LiDAR  data  and 
fiducial  locations  are  control 
data  sources,  followed  by 
operator  adjustment. 

Orthophotos 
aligned  to  +/- 
2  pixel  widths. 

(x,y  locations  of 

LiDAR  points)  -  (x,y 
locations  of  photo 
pixels)  at  target 
locations 

Orthophoto 
horizontal 
alignment  to 
fiducials 

Orthophotos  aligned 
to  survey  control 
points  so  that  target 
features  are  not 
displaced. 

Orthorectification  is  performed 
using  the  LiDAR  data  and 
fiducial  locations  are  control 
data  sources,  followed  by 
operator  adjustment. 

Orthophotos 
aligned  to  +/- 
3  pixel  widths. 

(x,y  locations  of  photo 
pixels)  -  (x,y  locations 
of  control  points)  at 
target  locations 

MRS 

identification* 

Correctly  identify 
all  previously 
identified  MRS. 

Identify  and  document  MRS 
from  LiDAR  and  orthophoto 
data  sets. 

All  MRS 

MRS  false 
alarm  rate 

No  areas  incorrectly 
identified  as  MRS. 

Identify  and  document  MRS 
from  LiDAR  and  orthophoto 
data  sets. 

All  MRS 

(#  of  MRS  previously 
identified)  -  (#  of  MRS 
identified  from 
demonstration  data) 

^  The  analysis  described  will  be  performed  for  each  separate  LiDAR  data  set  and  for  the  combined  LiDAR  data 
set,  and  for  each  orthophoto  data  set  and  for  the  combination  of  LiDAR  and  orthophotos.  The  “Desired  Resuh” 
metric  applies  to  the  combination  of  all  available  data.  It  is  currently  unknown  whether  the  metric  can  be 
achieved  for  some  portion  of  the  data  rather  than  the  combined  data  set. 
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Table  7-1)  Continued) 

Data  Quality  Metric  Calculation 


Analytical 

Objective 

Metric 

Action  to  Achieve  Metric 

Desired 

Result 

Calculation  Method 

MES 

boundary 

delineation 

Correctly  locate 

MRS  boundaries  to 
+/-  15%  of  ground- 
truthed  area. 

Identify  and  document  MRS 
boundaries  from  LiDAR  and 
orthophoto  data  sets  for  a 
selected  set  of  test  MRS. 

All  MRS 

(area  of  MRS 
previously  identified)  - 
(area  of  MRS  identified 
from  demonstration 
data),  plus  visual 
inspection 

MRS  feature 
identifieation 

Identify  features 
presenting  as 
human-made 
(anthropogenic)  not 
including  craters 
(e.g.  walls,  berms, 
pits,  small 
buildings). 

LiDAR  and  photo  data  sets  will 
be  examined  for  linear  features. 

90%  of 
features 
identified  from 
selected  field- 
identified 
features. 

(#  of  features 
previously  identified)  - 
(#  of  features  identified 
from  demonstration 
data) 

MRS  feature 
identifieation 

Identify  craters. 

Count  90%  of 
craters  over  1.5  m 
diameter  and  .3  m 
depth. 

Automated  algorithms  will  be 
used  to  identify  and  count  craters 
using  LiDAR  data. 

90%  of  craters 
identified 
outside  of 
crater  fields. 
95%  of  craters 
identified 
inside  crater 
fields. 

(#  of  craters  identified 
from  demonstration 
data)  -  (#  of  craters 
identified  through  field 
verification) 

MRS  feature 
identifieation 

Identify  vegetation 
patterns  indicating 
previous 
disturbance. 

Automated  algorithms  will  be 
used  to  map  vegetation  heights 
and  patterns  from  LiDAR  data. 
Results  will  be  examined  for 
linearity  or  other  regular  shapes. 
Orthophotos  will  be  examined 
for  regular  vegetation  patterns. 

[TBD] 

Visual  inspection 

MRS  feature 
identifieation 

Identify  established 
roads 

LiDAR  and  photo  data  sets  will 
be  examined  for  linear  features. 

100%  of 

established 

roads. 

Comparison  of  roads 
located  from 
demonstration  data  to 
roads  identified  by  on¬ 
site  inspection. 

MRS  feature 
identifieation 

Identify  vehicle 
tracks 

LiDAR  and  photo  data  sets  will 
be  examined  for  linear  features. 

90%  of  field- 
identified 
vehicle  tracks. 

Comparison  of  tracks 
located  from 
demonstration  data  to 
tracks  identified  by  on¬ 
site  inspection. 

MRS  feature 
identifieation 

Identify  topography 
that  can  limit  access 

LiDAR  data  sets  will  be  used  to 
map  areas  above  designated 
slope. 

100%  of  areas. 

Comparison  of  slope 
data  derived  from 
demonstration  data  with 
reports  of  on-site 
inspection. 
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7.2  HAND  CALCULATIONS 


Hand  calculations  will  be  legibly  reeorded  on  caleulation  sheets  and  in  a  logieal  progression, 
with  sufficient  deseriptions.  Major  ealeulations  will  be  checked  by  a  professional  at  a  level  equal 
to  or  higher  than  that  of  the  originator.  After  eompleting  the  check,  the  eheeker  will  sign  and 
date  the  ealeulation  sheet  immediately  below  the  signature  of  the  originator.  Both  the  originator 
and  eheeker  are  responsible  for  the  eorreetness  of  ealeulations.  A  ealeulation  sheet  eontains  the 
following,  at  a  minimum: 

•  Projeet  title  and  brief  deseription  of  the  task 

•  Task  number  and  date  performed 

•  Signature  of  person  who  performed  the  ealeulation 

•  Basis  for  ealeulation 

•  Assumptions  made  or  inherent  in  the  ealeulation 

•  Complete  referenee  for  eaeh  souree  of  input  data 

•  Methods  used  for  ealeulations 

•  Results  of  ealeulations,  elearly  annotated 


7.3  COMPUTER  ANALYSIS 

Computer  analysis  ineludes  the  use  of  models,  programs,  and  data  management  systems.  For 
published  software  with  existing  doeumentation,  test  ease  runs  are  periodieally  performed  to 
verify  that  the  software  is  performing  eorreetly.  Both  systematie  analysis  and  random  error 
analysis  are  investigated  and  appropriate  eorrective  action  measures  are  taken  as  needed. 
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8.0  QUALITYASSURANCE  AUDITS  AND  REPORTS 


8.1  STATUS  REPORTS 

Status  reporting  will  occur  throughout  the  project.  Status  reporting  is  designed  to  verify  and 
record  quality  assurance  activities  in  order  to  ensure  that  data  quality  meets  project 
specifications.  During  field  data  collection,  status  reporting  will  be  made  by  the  Operations 
Manager  in  charge  of  data  collection  for  TRSI;  these  reports  will  be  delivered  to  the  URS  PI  and 
the  URS  QAO.  During  data  processing  and  analysis  by  URS,  status  reporting  will  occur  during 
each  phase  of  the  analysis,  and  will  consist  primarily  of  URS  QA/QC  forms  documenting  the 
design  of  analysis  methods  and  review  of  resulting  products  for  compliance  with  project 
specifications.  These  reports  will  be  signed  by  the  Assistant  Investigator  and  delivered  to  the 
URS  PI  and  the  URS  QAO. 

Status  reporting  for  each  phase  of  the  project  will  take  place  as  follows: 

Pre-flight — A  report  will  be  made  to  URS  of  pre-flight  activities,  including  the  following: 

•  Survey  control  point(s)  established,  their  locations  and  ties  to  established  control 

•  Confirmation  of  locations  for  established  control  points. 

•  Results  of  calibration  flights  and  any  adjustments  made. 

•  Selection  and  documentation  of  sensor  parameters  for  low-level  LiDAR  flights,  and 
associated  planned  flight  lines. 

Data  Collection  Flights — A  daily  QA/QC  report  will  be  made  to  URS,  including  the  following 
items: 

•  GPS  solution  quality.  Results  of  daily  GPS  check. 

•  Flight  logs  showing  time,  orientation,  and  altitude  for  each  flight  line. 

•  Actual  flight  lines  from  GPS  records  during  each  flight. 

•  Area  coverage  for  LiDAR  and  images. 

•  LiDAR  point  density. 


INITIAL  DATA  PROCESSING  TO  CREATE  LIDAR  POINTS  AND  ORTHOPHOTO 
MOSAICS— A  QA/QC  REPORT  WILL  ACCOMPANY  DELIVERY  OF  PRIMARY 
DATA  TO  URS,  INCLUDING  THE  FOLLOWING  ITEMS: 

•  Calibration  factors  for  creation  of  LiDAR  points. 

•  Horizontal  and  vertical  positional  accuracies  for  LiDAR  points  and  orthophoto  pixels. 

•  Final  LiDAR  point  densities  for  each  LiDAR  flight. 
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Creation  of  Initial  GIS  Products — A  URS  QA/QC  form  will  be  completed  for  each  of  the 
following  steps: 

•  Setup  of  project  data  directories.  This  report  will  document  fde  locations,  fde  types,  data 
attribute  types,  and  compliance  with  EDD  standards. 

•  Finalizing  of  GIS-based  analysis  methods.  This  report  will  document  analysis  methods. 

•  Completion  of  each  major  phase  of  analysis,  documenting  results  and  comparison  to 
project  specifications. 

Delivery  of  Data  and  GIS  Products  to  ESTCP — A  QA/QC  report  will  accompany  all  data 
delivered  to  ESTCP,  documenting  review  of  data  deliverable  formats  and  compliance  with  EDD 
specifications. 


8.2  AUDITS 

Systems  and  performance  audits  and  surveillances  are  conducted  as  the  principal  means  to 
determine  compliance  with  the  project-specific  documents.  Audits  and  surveillances  are  used  to 
formally  review  individual  projects  during  their  course  and  across  all  levels  of  management.  The 
QAO  has  the  primary  responsibility  for  conducting  audits  and  surveillances,  portions  of  which 
may  be  delegated  to  an  auditing  team  of  senior  technical  specialists.  No  specific  audits  have 
been  planned  during  the  field  and  analysis  activities  due  to  their  limited  duration. 

Technical  specialists  must  be  familiar  with  the  technical  and  procedural  requirements  of  both 
field  and  office  operations,  as  well  as  the  associated  QA  plans.  In  addition,  auditors  may  not  be 
directly  involved  with  the  actual  tasks  themselves  to  ensure  no  introduction  of  bias  into  the 
auditing  process. 

An  audit  or  surveillance  may  be  initiated  (if  required)  prior  to  the  award  of  a  subcontract  to 
determine  the  capability  of  a  potential  subcontractor,  when  reorganization  or  major  revision  has 
been  made  to  the  project-specific  plans,  at  any  time  a  nonconformance  is  suspected,  or  to  verity 
that  corrective  actions  for  nonconformance  have  been  implemented. 

QA  audits  include  auditor  identification,  audit  notification,  audit  reporting,  identification  of 
nonconformances,  establishment  of  corrective  actions,  and  audit  completion  notification.  In 
circumstances  where  corrective  actions  have  not  been  completed  as  planned  or  scheduled,  the 
auditing  process  provides  for  management  intervention  to  resolve  problems  and  for  issuance  of 
stop  work  orders,  if  necessary. 
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Two  types  of  audits  may  be  performed,  a  performanee  audit  or  a  teehnieal  system  audit. 
Performanee  audits  will  be  eondueted  for  this  projeet  to  determine  the  status  and  effeetiveness  of 
field  data  eolleetion  and  to  provide  a  quantitative  measure  of  the  quality  of  data  generated.  Field 
performanee  will  be  evaluated  by  URS  through  review  of  the  LiDAR  and  orthophoto  data 
eompared  to  project  specifications.  This  review  will  include: 

•  Review  of  LiDAR  point  horizontal  and  vertical  locations  compared  to  survey  control 
point  locations. 

•  Review  of  LiDAR  point  density  for  each  flight  compared  to  project  specifications. 

•  Review  of  LiDAR  point  separation  to  create  bare-earth  point  sets  without  influence  of 
vegetation. 

•  Review  of  orthophoto  image  quality. 

•  Review  of  orthophoto  pixel  locations  in  comparison  to  LiDAR  and  survey  control  points. 

Technical  system  audits  are  used  to  confirm  the  adequacy  of  the  data  collection  (field  operation) 
and  data  generation  (laboratory  or  office  operation)  systems.  The  on-site  audits  are  conducted  to 
determine  whether  the  project-specific  plans  and  field  and  laboratory  SOPs  are  being  properly 
implemented.  Technical  audits  are  not  planned  for  this  project;  however,  if  substantial 
nonconformances  are  identified  or  if  there  is  concern  regarding  the  quality  of  data  and  related 
documentation,  then  technical  system  audits  will  be  employed. 


8.3  INDEPENDENT  TECHNICAL  REVIEW  AND  READINESS  REVIEW 

An  independent  technical  review  is  a  documented  critical  review  of  work  of  a  substantive  nature 
identified  as  a  deliverable.  These  reviews  will  be  conducted  by  experienced  and  qualified 
personnel  to  ensure  the  quality  and  integrity  of  tasks  and  products  by  allowing  the  work  and/or 
deliverable  to  undergo  objective,  critical  scrutiny.  For  this  demonstration,  independent  technical 
review  will  be  performed  by  ESTCP  or  is  designated  agents. 


8.4  CORRECTIVE  ACTIONS 

Evaluation  of  QC  data  and  review  of  audits  conducted  for  field  and  analysis  operations  may 
indicate  the  need  for  a  corrective  action.  Problems  arising  with  QA  data  will  be  addressed  by 
the  URS  QAO  through  communication  of  the  identified  problem  and  a  potential  corrective  action 
to  the  URS  PL  The  URS  PI  will  relay  this  information  to  the  project  staff  for  implementation. 
Project  staff  will  then  report  back  to  the  URS  PI  upon  successful  implementation  of  the 
corrective  action.  All  corrective  actions  are  required  to  be  documented.  This  documentation 
shall  include  at  a  minimum  the  description  of  the  problem,  when  the  problem  was  discovered, 
how  it  was  communicated,  and  specifics  of  the  response  action. 
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9.0  DATA  FORMAT  AND  STORAGE 


9.1  DATA  FORMAT 

Project  data  will  be  collected  primarily  by  sensors  in  electronic  formats.  Some  data,  however, 
will  be  collected  by  hand. 

Hand-collected  data  includes  GPS  survey  data  to  establish  control  points,  calibration  checklists, 
records  of  equipment  settings,  and  flight  logs.  All  hand  data  entries  will  be  made  and  dated  on 
the  day  of  entry  and  signed  or  initialed  by  the  person  entering  the  data.  Any  change  to  an  entry 
will  not  obscure  the  original  entry,  will  indicate  the  reason  for  the  change,  and  will  be  dated  and 
initialed  by  the  person  making  the  change. 

Electronic  data  will  be  collected  by  the  GPS,  IMU,  LiDAR  sensor  and  digital  camera.  The 
person  responsible  for  equipment  operation  for  each  flight  will  be  identified  and  recorded 
through  initials  on  the  field  data  collection  sheet  at  the  conclusion  of  each  flight.  Changes  are 
not  made  to  automated  data  entries,  which  are  preserved  in  their  original  form.  Update  of 
automated  data  is  accomplished  only  through  re-flight,  which  creates  a  new  data  set.  All 
electronic  data  contains  the  date  and  time  of  collection,  which  is  automatically  recorded.  Repeat 
data  sets  covering  the  same  area  can  be  distinguished  from  this  time  stamp,  however  all  external 
hard  drives  will  be  labeled  with  the  date  of  collection  and  the  project  number  and  area  name. 

Following  each  day’s  flight  missions,  data  will  be  reviewed  as  describe  above  to  verify  that  data 
has  been  accurately  recorded.  Unforeseen  circumstances  that  may  affect  the  integrity  of  the 
demonstration  include  weather  problems,  equipment  malfunction  and  failure  of  hard  drives  used 
for  data  storage.  Equipment  function  is  monitored  by  the  operator  and  through  automated 
systems  throughout  each  data  collection  flight,  and  the  impact  of  hard  drive  failure  is  minimized 
through  creating  a  duplicate  copy  of  all  data.  Weather  problems  will  be  addressed  through  re¬ 
scheduling  of  data  collection  activities  as  needed.  Unanticipated  problems  will  also  be  addressed 
through  scheduling  of  an  extra  flight  day  past  the  two  flight  days  needed  in  order  to  repeat 
unsatisfactory  flights. 


9.2  DATA  STORAGE  AND  ARCHIVING  PROCEDURES 

LiDAR  and  orthophotography  both  involve  collection  of  large  quantities  of  data,  which  limits  the 
usefulness  of  data  storage  methods  such  as  CDs  or  DVDs.  Large  project  areas  can  sometimes 
involve  too  much  data  for  network  backup  tapes,  although  this  is  not  expected  to  be  the  case  for 
the  Kirtland  PBR  demonstration.  The  principle  data  storage  method  consists  of  redundant 
external  hard  drives.  The  standard  URS  and  TRSI  procedure  is  as  follows: 

•  During  data  collection,  LiDAR  and  camera  image  data  is  recorded  directly  to  the  hard 
drive  of  the  sensor  package.  At  the  end  of  each  flight  this  data  is  copied  to  external  hard 
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drives.  Two  copies  are  made,  one  of  which  is  sent  to  the  TRSI  office  for  processing,  and 
one  of  which  is  retained  as  a  backup. 

•  During  data  processing  to  create  and  calibrate  the  LiDAR  points  orthophoto  mosaics, 
data  is  backed  up  nightly.  Backup  drives  are  stored  off-site.  [CONFIRM  OFF-SITE 
STORAGE] 

When  data  processing  has  been  completed,  data  is  transferred  to  the  URS  office  for  analysis. 
Transfer  is  either  by  external  hard  drive  or  ftp  transfer  depending  on  file  sizes.  A  copy  is 
retained  as  backup. 

At  the  URS  office,  data  is  copied  to  external  hard  drive  and  to  the  URS  corporate  network. 
During  processing  and  analysis,  files  are  backed  up  nightly;  backup  tapes  are  stored  off-site. 
Upon  completion  of  analysis,  data  will  be  transferred  to  ESTCP  and  a  copy  retained  on  external 
hard  drive  as  backup,  in  addition  to  the  corporate  network  backup  tapes. 

Data  availability  following  changes  in  key  personnel  is  accomplished  through  two  standard 
procedures.  Eirst,  a  file  naming  and  location  protocol  is  established  at  the  beginning  of  the 
project,  and  this  information  is  recorded  in  the  project  files.  Eile  naming  conventions,  which  are 
recorded  in  writing,  include  metadata  regarding  the  provenance  of  all  data  files  including 
primary  data,  intermediate  analytic  products,  and  final  products.  Second,  all  GIS  products 
include  metadata  that  will  meet  the  specifications  of  the  Electronic  Data  Deliverable  (EDD) 
specifications. 
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ATTACHMENT  A 


TECHNICAL  REVIEW  QA/QC  FORM 


Technical  Review  QA/QC  Form 

To  be  completed  by  technical  reviewers.  This  form  is  used  to  document  technical  review  of 
methods  and  products. 
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Munitions  and  Explosives  of  Concern 

PBR 

Precision  Bombing  Range 

PI 

Principal  Investigator 

SSR 

Site  Safety  Representative 

TRSI 

Terra  Remote  Sensing,  Inc. 

URS 

URS  Group,  Inc. 

URS  PI 

URS  Principal  Investigator 

IV 


1.0  INTRODUCTION 


This  document  serves  as  the  health  and  safety  plan  (HASP)  for  the  Environmental  Security 
Technology  Certification  Program  (ESTCP)  Demonstration  Project  entitled  “High  Density 
LiDAR  and  Orthophotography  in  UXO  Wide  Area  Assessment”,  to  be  performed  at  the  former 
Kirtland  Precision  Bombing  Range  (PBR),  New  Mexico  (Eigure  1-1).  This  HASP  has  been 
prepared  to  describe  the  procedures  that  will  be  implemented  to  manage  the  health  and  safety 
aspects  of  the  demonstration.  This  HASP  complies  with,  but  does  not  replace,  applicable 
Eederal,  State  and  local  regulations.  This  HASP  is  to  be  used  by  URS  and  subcontract  personnel 
as  a  supplement  to  these  rules,  regulations,  and  guidance.  Eor  URS  employees,  this  HASP  is  to 
be  augmented  by  applicable  provisions  of  the  Demonstration  Plan,  along  with  the  URS  Health 
and  Safety  Program  and  Management  System;  relevant  standards  from  that  program  and  system 
are  required  to  be  available  on  site  during  all  activities. 

Figure  1-1  Kirtland  AFB  PBS  Vicinity  Map 
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2.0  ORGANIZATION  SAFETY  RESPONSIBILITIES 


2.1  RESPONSIBILITIES 

The  following  subsections  describe  personnel  with  safety-related  duties  and  their  specific 
responsibilities. 

2.1.1  URS  Principle  Investigator  (PI) 

The  URS  PI  is  responsible  for  all  site  operations  and  project  implementation.  In  addition,  the  PI 
performs  the  following  duties: 

•  Ensures  adequate  resources  are  available  to  implement  and  carry  out  all  site  safety 
activities 

•  Ensures  that  all  personnel  assigned  to  the  site  have  received  the  necessary  health 
and  safety  training 

•  Eamiliarizes  all  on-site  personnel  with  the  site  safety  requirements 

•  Assigns  key  safety  duties  and  responsibilities  to  team  members 

•  Ensures  that  all  necessary  personal  protective  equipment  (PPE)  is  available  and 
on  site. 


2.1.2  Subcontractor  Safety  Representative 

Each  subcontractor  is  requested  to  designate  a  Subcontractor’s  Safety  Representative  (SSR)  who 
is  the  subcontractor  supervisor.  URS  will  consider  the  SSRs  responsible  for  the  safe  and 
healthful  performance  of  work  by  their  work  force  and  subcontractors.  During  subcontractor 
activities  on  site,  the  SSR  will  perform  continuing  work  area  inspections,  and  conduct  safety 
meetings  and  safety  orientations  for  all  his  subcontractor  employees.  The  SSR  will  attend  all 
safety  meetings  and  will  also  investigate  accidents  and  overexposures  involving  subcontractor 
personnel. 


2.1.3  TRSI 

Safety  is  an  integral  part  of  Terra  Remote  Sensing  Inc.'s  (TRSI)  day-to-day  activities.  It  is  our 
first  priority  in  the  performance  of  our  work.  The  protection  of  people,  equipment,  property  and 
the  public  is  an  attitude  that  is  emphasized  and  rewarded  in  all  of  our  operations. 
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The  economic  benefits  of  working  safely  far  outweigh  the  costs  of  non-compliance.  The  greatest 
cost  is  the  human  cost.  By  protecting  our  employees,  TRSI  is  also  protecting  their  friends, 
families,  fellow  workers,  management,  the  public  and  the  environment. 

Protection  of  one's  livelihood  is  first  and  foremost  with  our  safety  program.  This  program  will 
also  contribute  to  employee  morale  and  pride  because  of  their  involvement  in  identifying  safety 
needs  and  developing  safe  work  procedures. 

TRSI  will  ensure  that  recognized  health  and  safety  standards  and  legal  requirements  are  met 
through  the  provision  of  adequate  facilities,  equipment,  procedures,  training  and  management 
systems. 

Everyone  employed  by  TRSI  is  responsible  for  maintaining  our  safety  program.  Supervisors  and 
managers  are  responsible  for  identifying  safety  needs,  communicating  safety  hazards, 
investigating  hazardous  conditions  and  accidents,  providing  training,  supplying  or  wearing 
appropriate  safety  and  personal  protective  equipment,  and  ensuring  all  equipment  is  properly 
maintained  and  meets  legislated  safety  standards.  Their  role  is  supported  by  input  from  all 
employees. 

All  company  employees,  contractors  and  subcontractors  on  company  worksites  are  responsible 
for  obeying  all  safety  rules,  following  recommended  safe  work  procedures,  wearing  and  using 
personal  protective  equipment  when  required,  participating  in  safety  training  programs  and 
informing  party  chiefs  of  any  unsafe  work  conditions.  Everyone  has  the  right  and  responsibility 
to  refuse  to  do  work  when  unsafe  conditions  exist. 

TRSI  takes  the  approach  that  safety,  and  all  of  the  implications  of  a  health  and  safety  program 
are  part  of  an  overall  attitude  expressed  by  all  of  its  employees,  supervisors  and  managers.  An 
open  and  ongoing  dialogue  on  all  aspects  of  loss  prevention  is  paramount  to  the  well  being  of  all 
concerned.  By  fulfilling  our  safety  responsibilities,  everyone  who  works  for  our  company  will 
share  the  benefits  of  a  safe  workplace. 


2,1,4  AeroWest  Helicopters 

AeroWest  Helicopters,  Inc.  is  a  full-service  helicopter  company  based  at  the  Double  Eagle 
airport  on  the  demonstration  site.  AeroWest  is  regulated  under  EAR  135,  Operating 
Requirements:  Commuter  and  On-Demand  Operations  and  Rules  Governing  Persons  On  Board 
Such  Aircraft,  promulgated  by  the  Eederal  Aviation  Administration.  AeroWest  is  certified  under 
these  regulations;  a  copy  of  the  certification  is  attached  and  further  information  is  available  upon 
request.  Additional  material  submitted  by  AeroWest  includes  that  the  company  flies  an  average 
of  2500  hours  per  year  and  has  never  had  a  passenger  hurt  since  operations  were  initiated  in 
1984. 
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Table  2-1. 

Project  Safety  Contacts 


Key  Role 

Name 

Telephone 

On-Site  Emergencies 

Any  emergency 

911 

Ambulance 

911 

Kirtland  PER  /  Double  Eagle  Airport  Manager 

(505)  244-7888 

Fire 

911  or 

University  of  New  Mexico  Hospital,  Albuquerque 

(505)  272-2111 

URS  1 

URS  Principal  Investigator 

Tom  Tomczyk 

(206)  438-2137 

URS  Health  and  Safety  Manager 

Mark  Litzinger 

(206)  438-2199 

TRSI  1 

TRSI  Site  Safety  Officer 

(xxx)  xxx-xxxx 

*  In  the  event  of  an  emergency,  the  URS  PI  and  the  ESTCP  PM  are  to  be  notified  immediately. 
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3.0  PERSONNEL  MEDICAL  QUALIFICATIONS 
AND  TRAINING  REQUIREMENTS 


Team  members  performing  eritieal  funetions,  sueh  as  pilot,  must  have  a  physical  examination 
prior  to  participation  in  field  activities  to  verify  that  the  worker  is  capable  of  performing  those 
duties,  and  is  free  of  medical  conditions  that  may  be  aggravated  while  performing  those  duties. 
All  site  personnel  will  participate  in  an  initial  safety  briefing,  followed  by  daily  briefings  to 
discuss  the  effectiveness  of  controls  and  overall  project  safety.  All  necessary  certifications  and 
licenses  will  be  produced  prior  to  mobilization  and  a  copy  of  the  documentation  will  be  kept  on 
site. 
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4.0  SITE  HAZARD  ANALYSIS 


This  section  describes  the  physical  hazards  that  may  be  encountered  during  site  operations. 


4.1  LASER 

TRSI  uses  a  class  4  laser  in  their  aerial  survey  equipment  MPE  (Maximum  Permissible 
Exposure)  and  NHZ  (Nominal  Hazard  Zone)  have  been  calculated  and  demonstrate  that  the 
current  laser  is  a  potential  hazard  during  indoor  scenarios  only.  Indoor  control  measures  have 
been  implemented  to  reduce  the  possibility  of  exposure  to  the  beam. 

During  the  intended  operation  (outdoors)  of  the  survey  laser,  the  laser  beam  is  fully  enclosed 
except  at  the  scanning  mirror  that  disperses  the  laser  down  onto  the  earth's  surface.  The  remote 
sensing  pod  is  located  on  the  underside  of  the  aircraft  where  access  is  not  possible  during  data 
collection.  Therefore,  immediately  at  the  output  aperture,  the  NHZ  computation  is  not  applicable. 
Eor  distal  outdoor  targets,  NHZ  is  not  applicable  as  the  beam  is  collimated  at  high  altitude.  Plus, 
the  likelihood  is  very  small  that  a  subject  on  the  ground  will  be  exposed  to  more  than  one  pulse 
of  the  laser  because  the  laser  is  reflected  in  multiple  directions  at  a  high  rate  combined  with  the 
velocity  of  the  aircraft.  As  standard  safety  precautions,  the  scanner  is  always  running  prior  to 
laser  initiation  and  the  laser  is  not  energized  until  the  aircraft  carrying  the  unit  is  100m  above 
ground. 


4.2  AIRCRAFT 

TRSI’s  aerial  survey  process  requires  the  use  of  a  helicopter  or  fixed  wing  aircraft.  These 
aircrafts  are  chartered  and  the  responsibility  of  flying  the  aircraft  is  with  the  subcontractor. 
Hazards  associated  with  these  transportation  vehicles  ranges  from  Class  I  to  III  and  are:  crash 
resulting  in  injury  or  fatality,  struck  by  rotor  or  propeller,  noise,  stranded  in  remote  location,  hot 
exhaust,  improper  storage  of  equipment.  TRSI  employees  working  around  aircraft  follow  general 
helicopter  and  fixed  wing  safety  rules  and  guidelines,  and  follow  instructions  from  the  contracted 
pilot. 


4.3  OTHER  POTENTIAL  HAZARDS 

MEC.  MEC  may  be  present  in  the  study  area.  Survey  personnel  are  not  expected  to  be  on  the 
ground  outside  of  the  airport  for  extended  periods.  If  personnel  do  encounter  MEC,  avoid  the 
item  and  contact  the  Site  Safety  Officer. 

Heat  Stress.  Heat  stress  is  considered  an  issue  when  the  ambient  temperature  exceeds  70°E. 
Personnel  are  expected  to  keep  hydrated  by  consistent  intake  of  fluids,  especially  water.  If 
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5.0  EMERGENCY  RESPONSE 


Injuries  that  occur  at  job  sites  must  be  handled  quickly  and  competently.  When  serious  injuries, 
breathing  difficulties,  intense  pain,  or  unconsciousness  occur,  site  personnel  will  immediately 
seek  help  from  professional  paramedics  using  a  building  phone  or  cellular  phone  and  dialing 
911.  For  non-life-threatening  injuries  that  do  not  impair  driving  ability,  site  personnel  will  drive 
to  the  hospital.  If  driving  ability  is  affected  by  the  injury,  site  personnel  will  use  the  designated 
phone  to  call  911  for  assistance. 

A  first  aid  kit  will  be  available  at  the  site  to  treat  minor  injuries.  First  aid  responders  should 
protect  themselves  from  contact  with  blood  and  other  human  body  fluids  by  wearing  latex  gloves 
or  establishing  an  equivalent  barrier.  Antiseptic  wipes  and  latex  gloves  should  be  carried  in  the 
first  aid  kit.  First  aid  responder’s  hands  should  be  washed  with  hot,  soapy  water  as  soon  as 
possible  after  first  aid  treatment  is  administered.  All  injuries  will  be  reported  to  the  PI  as  soon  as 
possible. 


After  making  initial  contact  with  PI,  and  within  24  hours  of  a  serious  injury  of  fatality,  a 
Contractor  Significant  Incident  Report  (CSIR)  must  be  submitted.  If  the  mishap  is  a  lost-time  or 
a  recordable  incident,  submit  the  CSIR  within  5  days  of  the  incident. 
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symptoms  of  heat  stress  are  observed,  immediate  rest  and  monitoring  of  the  individual  is 
imperative. 

Traffic.  Air  and  vehicle  traffic  at  the  site  is  expected  to  be  moderate  to  heavy  and  personnel 
should  exercise  caution  at  the  airport  and  on  or  near  roadways.  Communication  with  air  traffic 
controllers  and  ground  crews  is  vitally  important  and  will  be  constantly  monitored. 

Noise.  High  noise  levels  may  be  encountered  during  flight  operations.  Noise  and  hearing 
conservation  practices  will  be  in  effect. 

Slip-Trip-Fall  Hazards.  Care  will  be  taken  to  ensure  proper  footing  and  handholds,  particularly 
if  wet  conditions  make  the  ground  surface  or  hand/sampling  tools  slippery.  The  sampling  area 
will  be  kept  uncluttered.  Good  housekeeping  around  equipment  will  be  enforced. 

Back  Injuries.  Workers  will  use  proper  lifting  techniques,  lifting  with  the  legs  and  not  the  back. 
Lifting  loads  over  50  pounds  require  a  second  person  or  mechanical  device.  Whenever  possible, 
mechanical  devices  should  be  used  to  lift  or  move  heavy  loads. 

Hand  Tools.  Eye  injuries,  puncture  wounds,  cuts,  or  lacerations  could  result  from  use  of  hand 
tools  while  collecting  samples  or  repairing  equipment.  Tools  should  be  in  good  condition  and 
the  right  tool  should  be  selected  for  the  job.  Safety  glasses  with  side  shields  or  safety  goggles 
should  be  worn  whenever  projectiles  are  a  potential  problem.  Loose  clothes  or  jewelry  will  not 
be  worn  because  they  could  get  caught  in  moving  equipment.  Steel-toed  shoes  should  be  worn 
when  there  is  any  risk  that  something  could  fall  on  the  foot.  Tools  should  be  stored  safely,  with 
sharp  edges  protected. 

Screwdrivers  should  not  be  used  as  chisels,  because  their  tips  could  break  or  fly  off  The  head 
could  fly  off  a  hammer  with  a  loose  or  cracked  wooden  handle.  Impact  tools  such  as  chisels  or 
wedges  with  mushroomed  heads  might  shatter  on  impact,  sending  sharp  fragments  flying. 

Knives,  saw  blades,  and  scissors  must  be  sharp. 


7 


APPENDIX  I 
Points  of  Contact 


FINAL  REPORT 

High  Density  Lidar  and  Orthophotography 
in  UXO  Wide  Area  Assessment 
Contraet  No.  N44255-02-D-2008 


Appendix  I 
Revision  No.:  0 
Date:  August  2007 
Page  I-l 


POINTS  OF  CONTACT 


Name 

Organization  and  Address 

Phone/F  ax/email 

Role  in  Project 

Jeffrey  Marqusee 

ESTCP 

901  North  Stuart  St. 

Suite  303 

Arlington,  VA  22203 

703-696-2120 

703-696-2114 

Director,  ESTCP 

Anne  Andrews 

ESTCP 

901  North  Stuart  St. 

Suite  303 

Arlington,  VA  22203 

703-696-3826 

703-696-2114 

Program  Manager, 

Munitions 

Management 

Herb  Nelson 

Naval  Researeh  Lab 

1155  Herndon  Parkway 

Suite  900 

Herndon,  WA  20170 

202-767-3686 

202-404-8119 

Herb.nelson@nrl.nav 

y.mil 

Project  Manager, 
ESTCP  Wide  Area 
Assessment  Pilot 
Program 

Dale  Bennett 

URS 

1501  4th  Ave 

Suite  1400 

Seattle  WA  98101-1616 

206-438-2026 
206-438-2699 
dale  bennett@ 
urseorp.eom 

Data  Evaluation 
and  GIS  Lead 

Riek  Quinn 

TRSI 

1962  Mills  Road 

Sidney  BC 

Canada  V8L  5Y3 

250-656-0931 

250-656-4604 

riek.quinn@ 

terraremote.eom 

Terra  Remote 
Sensing  Principle 
in  Charge 

Jim  Vosberg 

TRSI 

1962  Mills  Road 

Sidney  BC 

Canada  V8L  5Y3 

250-656-0931 

250-656-4604 

jim.vosberg@ 

terraremote.eom 

Terra  Remote 
Sensing  Operations 
Manager 

Bill  Rohrer 

URS 

1501  4th  Ave 

Suite  1400 

Seattle  WA  98101-1616 

206-438-2296 
206-438-2699 
bill  rohrer@ 
urseorp.eom 
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